[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:32:19 dist-test-slave-gkw7 krb5kdc[27818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:32:19 dist-test-slave-gkw7 krb5kdc[27818](info): set up 2 sockets
Aug 09 03:32:19 dist-test-slave-gkw7 krb5kdc[27818](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 03:32:20.954124 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:32:21 dist-test-slave-gkw7 krb5kdc[27818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:32:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:32:21.081735 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:43703
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:43703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:21.167696 27840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:21.167964 27840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:21.168001 27840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:21.169188 27840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:21.169261 27840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:21.169284 27840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:21.169319 27840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:21.169366 27840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:32:21.170946 27840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:43703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:43703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:21.171252 27840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:21.171494 27840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:21.176784 27855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:21.176852 27858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:21.176796 27856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:21.176833 27840 server_base.cc:1034] running on GCE node
I20260809 03:32:21.177286 27840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:21.177675 27840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:21.178850 27840 hybrid_clock.cc:648] HybridClock initialized: now 1786246341178851 us; error 117 us; skew 500 ppm
Aug 09 03:32:21 dist-test-slave-gkw7 krb5kdc[27818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246341, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:21.181435 27840 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:32:21.181944 27840 webserver.cc:460] Webserver started at http://127.27.40.190:38555/ using document root <none> and password file <none>
I20260809 03:32:21.182183 27840 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:21.182250 27840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:21.182403 27840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:21.183542 27840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "d2a98e0bc40541a79c22d39f7a7c3239"
format_stamp: "Formatted at 2026-08-09 03:32:21 on dist-test-slave-gkw7"
server_key: "d1e8ce8905951db675dd49f134f912dd"
server_key_iv: "6f0c4b1296ed26ee7640431c73f98cd7"
server_key_version: "encryptionkey@0"
I20260809 03:32:21.183923 27840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "d2a98e0bc40541a79c22d39f7a7c3239"
format_stamp: "Formatted at 2026-08-09 03:32:21 on dist-test-slave-gkw7"
server_key: "d1e8ce8905951db675dd49f134f912dd"
server_key_iv: "6f0c4b1296ed26ee7640431c73f98cd7"
server_key_version: "encryptionkey@0"
I20260809 03:32:21.185328 27840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:32:21.186144 27866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:21.186335 27840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 03:32:21.186439 27840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "d2a98e0bc40541a79c22d39f7a7c3239"
format_stamp: "Formatted at 2026-08-09 03:32:21 on dist-test-slave-gkw7"
server_key: "d1e8ce8905951db675dd49f134f912dd"
server_key_iv: "6f0c4b1296ed26ee7640431c73f98cd7"
server_key_version: "encryptionkey@0"
I20260809 03:32:21.186518 27840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:21.208593 27840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:21.210953 27840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:21.211120 27840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:21.215662 27840 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:43703
I20260809 03:32:21.215703 27936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:43703 every 8 connection(s)
I20260809 03:32:21.216075 27840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:21.216967 27937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:21.217887 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 27840
I20260809 03:32:21.217971 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:21.218583 27810 external_mini_cluster.cc:1428] Setting key fbc2e4a32fbf379c5ff763db1ed338f7
I20260809 03:32:21.220041 27937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239: Bootstrap starting.
I20260809 03:32:21.220763 27937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:21.221673 27937 log.cc:824] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:21.222465 27937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239: No bootstrap required, opened a new log
I20260809 03:32:21.224900 27937 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a98e0bc40541a79c22d39f7a7c3239" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 43703 } }
I20260809 03:32:21.225063 27937 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:21.225152 27937 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2a98e0bc40541a79c22d39f7a7c3239, State: Initialized, Role: FOLLOWER
I20260809 03:32:21.225327 27937 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a98e0bc40541a79c22d39f7a7c3239" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 43703 } }
I20260809 03:32:21.225426 27937 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:21.225471 27937 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:21.225517 27937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:21.226256 27937 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a98e0bc40541a79c22d39f7a7c3239" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 43703 } }
I20260809 03:32:21.226384 27937 leader_election.cc:302] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2a98e0bc40541a79c22d39f7a7c3239; no voters:
I20260809 03:32:21.226590 27937 leader_election.cc:288] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:21.226625 27943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:21.226748 27943 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [term 1 LEADER]: Becoming Leader. State: Replica: d2a98e0bc40541a79c22d39f7a7c3239, State: Running, Role: LEADER
I20260809 03:32:21.226869 27937 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:21.226898 27943 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a98e0bc40541a79c22d39f7a7c3239" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 43703 } }
I20260809 03:32:21.227422 27946 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2a98e0bc40541a79c22d39f7a7c3239. Latest consensus state: current_term: 1 leader_uuid: "d2a98e0bc40541a79c22d39f7a7c3239" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a98e0bc40541a79c22d39f7a7c3239" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 43703 } } }
I20260809 03:32:21.227571 27944 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2a98e0bc40541a79c22d39f7a7c3239" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a98e0bc40541a79c22d39f7a7c3239" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 43703 } } }
I20260809 03:32:21.227603 27946 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:21.227638 27944 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239 [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:21.228463 27951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:21.229118 27951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:21.231011 27951 catalog_manager.cc:1347] Generated new cluster ID: c52475d56fe844b49b43fa2e599eda90
I20260809 03:32:21.231069 27951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 03:32:21 dist-test-slave-gkw7 krb5kdc[27818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:21.238435 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:21.219539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53172 (local address 127.27.40.190:43703)
0809 03:32:21.219701 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:21.219708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:21.219987 (+ 279us) server_negotiation.cc:407] Connection header received
0809 03:32:21.222598 (+ 2611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:21.222606 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:21.222668 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:21.222812 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:21.224576 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.225985 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.227356 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.228430 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.228544 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:21.233681 (+ 5137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:21.233685 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:21.233685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:21.233696 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:21.236123 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:21.236704 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:21.236707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:21.236708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:21.236812 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:21.237144 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:21.237145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:21.237145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:21.237430 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:21.237712 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:21.238010 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 03:32:21.238199 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260809 03:32:21.248360 27951 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:21.248896 27951 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:21.252853 27951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2a98e0bc40541a79c22d39f7a7c3239: Generated new TSK 0
I20260809 03:32:21.253021 27951 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:32:21.282689 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:43703
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:21.399152 27974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:21.399361 27974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:21.399386 27974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:21.400620 27974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:21.400671 27974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:21.400698 27974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:32:21.402402 27974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:43703
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:21.402681 27974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:21.402972 27974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:21.408680 27989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:21.408640 27982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:21.408638 27981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:21.408787 27974 server_base.cc:1034] running on GCE node
I20260809 03:32:21.409158 27974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:21.409405 27974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:21.410585 27974 hybrid_clock.cc:648] HybridClock initialized: now 1786246341410562 us; error 38 us; skew 500 ppm
Aug 09 03:32:21 dist-test-slave-gkw7 krb5kdc[27818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246341, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:21.413724 27974 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:32:21.414162 27974 webserver.cc:460] Webserver started at http://127.27.40.129:45935/ using document root <none> and password file <none>
I20260809 03:32:21.414404 27974 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:21.414451 27974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:21.414623 27974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:21.415792 27974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "099ee8d12a4a4452b63cb89374e2a36c"
format_stamp: "Formatted at 2026-08-09 03:32:21 on dist-test-slave-gkw7"
server_key: "d3a0112c9a46f90968c8c813afbe927c"
server_key_iv: "50ddff222a7df937bbeed14c5d934579"
server_key_version: "encryptionkey@0"
I20260809 03:32:21.416193 27974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "099ee8d12a4a4452b63cb89374e2a36c"
format_stamp: "Formatted at 2026-08-09 03:32:21 on dist-test-slave-gkw7"
server_key: "d3a0112c9a46f90968c8c813afbe927c"
server_key_iv: "50ddff222a7df937bbeed14c5d934579"
server_key_version: "encryptionkey@0"
I20260809 03:32:21.417809 27974 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:32:21.418627 27998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:21.418831 27974 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:21.418927 27974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "099ee8d12a4a4452b63cb89374e2a36c"
format_stamp: "Formatted at 2026-08-09 03:32:21 on dist-test-slave-gkw7"
server_key: "d3a0112c9a46f90968c8c813afbe927c"
server_key_iv: "50ddff222a7df937bbeed14c5d934579"
server_key_version: "encryptionkey@0"
I20260809 03:32:21.419003 27974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:21.429626 27974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:21.431835 27974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:21.431975 27974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:21.432250 27974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:21.432564 27974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:21.432612 27974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:21.432638 27974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:21.432653 27974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:21.445479 27974 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:42487
I20260809 03:32:21.445647 28172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:42487 every 8 connection(s)
I20260809 03:32:21.445994 27974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:32:21.448869 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 27974
I20260809 03:32:21.448952 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:32:21.449136 27810 external_mini_cluster.cc:1428] Setting key f98a3b06b06cd32342e2e2398594b856
Aug 09 03:32:21 dist-test-slave-gkw7 krb5kdc[27818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246341, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:21.456414 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:21.446838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:36105 (local address 127.27.40.190:43703)
0809 03:32:21.446923 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:21.446925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:21.447205 (+ 280us) server_negotiation.cc:407] Connection header received
0809 03:32:21.447659 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:21.447661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:21.447672 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:21.447736 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:21.448782 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.449441 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.450563 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.450788 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.450863 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:21.453323 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:21.453324 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:21.453324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:21.453342 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:21.454852 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:21.455373 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:21.455374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:21.455374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:21.455430 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:21.455725 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:21.455726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:21.455726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:21.455830 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:21.455878 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:21.456269 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 03:32:21.456348 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:32:21.456785 28179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:21.446784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:43703 (local address 127.27.40.129:36105)
0809 03:32:21.447012 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:21.447048 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:21.447565 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:21.447817 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:21.447821 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:21.448458 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:21.448654 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:21.448660 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.449519 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.449520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:21.450452 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:21.450454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.450885 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.450887 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:21.451026 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:21.451608 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:21.451616 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:21.453204 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:21.454932 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:21.454940 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:21.454943 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:21.455290 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:21.455496 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:21.455497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:21.455497 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:21.455652 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:21.455895 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:21.455902 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:21.456087 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 03:32:21.456232 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 03:32:21.456431 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":145,"threads_started":1}
I20260809 03:32:21.457507 28174 heartbeater.cc:347] Connected to a master server at 127.27.40.190:43703
I20260809 03:32:21.457643 28174 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:21.457855 28174 heartbeater.cc:511] Master 127.27.40.190:43703 requested a full tablet report, sending...
I20260809 03:32:21.458495 27889 ts_manager.cc:194] Registered new tserver with Master: 099ee8d12a4a4452b63cb89374e2a36c (127.27.40.129:42487)
I20260809 03:32:21.459146 27889 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:36105
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:32:21.478765 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:43703
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:21.595340 28184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:21.595563 28184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:21.595593 28184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:21.596752 28184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:21.596794 28184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:21.596823 28184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:32:21.598457 28184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:43703
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:21.598654 28184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:21.598835 28184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:21.605331 28195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:21.605355 28190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:21.605409 28191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:21.606017 28184 server_base.cc:1034] running on GCE node
I20260809 03:32:21.606257 28184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:21.606514 28184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:21.607710 28184 hybrid_clock.cc:648] HybridClock initialized: now 1786246341607682 us; error 38 us; skew 500 ppm
Aug 09 03:32:21 dist-test-slave-gkw7 krb5kdc[27818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246341, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:21.611127 28184 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:32:21.611577 28184 webserver.cc:460] Webserver started at http://127.27.40.130:36745/ using document root <none> and password file <none>
I20260809 03:32:21.611846 28184 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:21.611929 28184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:21.612087 28184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:21.613158 28184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "fe88129c8f364ca5ba1444d0df4cac0d"
format_stamp: "Formatted at 2026-08-09 03:32:21 on dist-test-slave-gkw7"
server_key: "911546eaade6528ee1e8b0b6a32e983f"
server_key_iv: "a89ea25b0d4e6ce8cd2a81987a34aa8d"
server_key_version: "encryptionkey@0"
I20260809 03:32:21.613648 28184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "fe88129c8f364ca5ba1444d0df4cac0d"
format_stamp: "Formatted at 2026-08-09 03:32:21 on dist-test-slave-gkw7"
server_key: "911546eaade6528ee1e8b0b6a32e983f"
server_key_iv: "a89ea25b0d4e6ce8cd2a81987a34aa8d"
server_key_version: "encryptionkey@0"
I20260809 03:32:21.615227 28184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:32:21.616159 28204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:21.616365 28184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:21.616465 28184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "fe88129c8f364ca5ba1444d0df4cac0d"
format_stamp: "Formatted at 2026-08-09 03:32:21 on dist-test-slave-gkw7"
server_key: "911546eaade6528ee1e8b0b6a32e983f"
server_key_iv: "a89ea25b0d4e6ce8cd2a81987a34aa8d"
server_key_version: "encryptionkey@0"
I20260809 03:32:21.616554 28184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:21.624437 28184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:21.627278 28184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:21.627516 28184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:21.627897 28184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:21.628266 28184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:21.628325 28184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:21.628366 28184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:21.628396 28184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:21.640848 28184 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:39545
I20260809 03:32:21.640906 28369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:39545 every 8 connection(s)
I20260809 03:32:21.641273 28184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:32:21.645437 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 28184
I20260809 03:32:21.645566 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:32:21.645780 27810 external_mini_cluster.cc:1428] Setting key bb3f6cc087cc78a4cbc29a9c8904b215
Aug 09 03:32:21 dist-test-slave-gkw7 krb5kdc[27818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246341, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:21.652630 28373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:21.642092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:43703 (local address 127.27.40.130:48259)
0809 03:32:21.642370 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:21.642409 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:21.642939 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:21.643248 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:21.643252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:21.643830 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:21.644027 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:21.644032 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.644970 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.644971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:21.646195 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:21.646198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.646725 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.646727 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:21.646878 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:21.647487 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:21.647496 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:21.649017 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:21.651135 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:21.651141 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:21.651143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:21.651446 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:21.651699 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:21.651700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:21.651700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:21.651800 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:21.652146 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:21.652151 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:21.652268 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 03:32:21.652351 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 03:32:21.652470 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":164,"threads_started":1}
I20260809 03:32:21.652648 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:21.642133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:48259 (local address 127.27.40.190:43703)
0809 03:32:21.642195 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:21.642197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:21.642513 (+ 316us) server_negotiation.cc:407] Connection header received
0809 03:32:21.643054 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:21.643055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:21.643068 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:21.643122 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:21.644284 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.644853 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.646325 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.646615 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.646711 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:21.649122 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:21.649124 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:21.649124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:21.649134 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:21.651038 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:21.651533 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:21.651535 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:21.651535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:21.651611 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:21.651871 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:21.651873 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:21.651873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:21.652046 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:21.652161 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:21.652416 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 03:32:21.652525 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 03:32:21.653450 28371 heartbeater.cc:347] Connected to a master server at 127.27.40.190:43703
I20260809 03:32:21.653594 28371 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:21.653831 28371 heartbeater.cc:511] Master 127.27.40.190:43703 requested a full tablet report, sending...
I20260809 03:32:21.654301 27889 ts_manager.cc:194] Registered new tserver with Master: fe88129c8f364ca5ba1444d0df4cac0d (127.27.40.130:39545)
I20260809 03:32:21.654675 27889 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:48259
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:32:21.674539 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:43703
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:21.763329 28377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:21.763566 28377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:21.763616 28377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:21.764779 28377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:21.764845 28377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:21.764891 28377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:32:21.766448 28377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:43703
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:21.766745 28377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:21.766976 28377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:21.772131 28384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:21.772107 28387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:21.772092 28390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:21.772512 28377 server_base.cc:1034] running on GCE node
I20260809 03:32:21.772760 28377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:21.773010 28377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:21.774238 28377 hybrid_clock.cc:648] HybridClock initialized: now 1786246341774213 us; error 40 us; skew 500 ppm
Aug 09 03:32:21 dist-test-slave-gkw7 krb5kdc[27818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246341, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:21.776996 28377 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:32:21.777470 28377 webserver.cc:460] Webserver started at http://127.27.40.131:38787/ using document root <none> and password file <none>
I20260809 03:32:21.777747 28377 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:21.777822 28377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:21.777971 28377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:21.779014 28377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "b82c0bc829ae48cc941c9362bbf02a52"
format_stamp: "Formatted at 2026-08-09 03:32:21 on dist-test-slave-gkw7"
server_key: "cb7ca3535ca439fff00344bedae1795b"
server_key_iv: "c7296beb4d9bdc7c32b82128a979646e"
server_key_version: "encryptionkey@0"
I20260809 03:32:21.779390 28377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "b82c0bc829ae48cc941c9362bbf02a52"
format_stamp: "Formatted at 2026-08-09 03:32:21 on dist-test-slave-gkw7"
server_key: "cb7ca3535ca439fff00344bedae1795b"
server_key_iv: "c7296beb4d9bdc7c32b82128a979646e"
server_key_version: "encryptionkey@0"
I20260809 03:32:21.780820 28377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:32:21.781677 28398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:21.781867 28377 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:21.781970 28377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "b82c0bc829ae48cc941c9362bbf02a52"
format_stamp: "Formatted at 2026-08-09 03:32:21 on dist-test-slave-gkw7"
server_key: "cb7ca3535ca439fff00344bedae1795b"
server_key_iv: "c7296beb4d9bdc7c32b82128a979646e"
server_key_version: "encryptionkey@0"
I20260809 03:32:21.782052 28377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:21.789665 28377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:21.791548 28377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:21.791706 28377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:21.791929 28377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:21.792243 28377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:21.792299 28377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:21.792338 28377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:21.792366 28377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:21.799571 28377 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:34025
I20260809 03:32:21.799613 28549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:34025 every 8 connection(s)
I20260809 03:32:21.799978 28377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:32:21.800384 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 28377
I20260809 03:32:21.800462 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:32:21.800658 27810 external_mini_cluster.cc:1428] Setting key e1568979768e13d5da296e94f0cb5371
Aug 09 03:32:21 dist-test-slave-gkw7 krb5kdc[27818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246341, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:21.809605 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:21.800862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:52105 (local address 127.27.40.190:43703)
0809 03:32:21.800915 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:21.800918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:21.801285 (+ 367us) server_negotiation.cc:407] Connection header received
0809 03:32:21.801953 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:21.801954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:21.801968 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:21.802031 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:21.803023 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.803517 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.804497 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.804677 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.804729 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:21.806992 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:21.806993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:21.806994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:21.807000 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:21.808206 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:21.808575 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:21.808576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:21.808576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:21.808629 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:21.808847 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:21.808848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:21.808848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:21.808949 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:21.809001 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:21.809380 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 03:32:21.809466 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 03:32:21.809780 28552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:21.800878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:43703 (local address 127.27.40.131:52105)
0809 03:32:21.801052 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:21.801138 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:21.801840 (+ 702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:21.802101 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:21.802113 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:21.802703 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:21.802943 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:21.802948 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.803595 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.803596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:21.804425 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:21.804427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.804749 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.804750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:21.804857 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:21.805457 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:21.805466 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:21.806914 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:21.808284 (+ 1370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:21.808291 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:21.808292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:21.808510 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:21.808687 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:21.808688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:21.808688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:21.808792 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:21.809033 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:21.809040 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:21.809201 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 03:32:21.809312 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 03:32:21.809451 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
I20260809 03:32:21.810377 28550 heartbeater.cc:347] Connected to a master server at 127.27.40.190:43703
I20260809 03:32:21.810469 28550 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:21.810685 28550 heartbeater.cc:511] Master 127.27.40.190:43703 requested a full tablet report, sending...
I20260809 03:32:21.811167 27889 ts_manager.cc:194] Registered new tserver with Master: b82c0bc829ae48cc941c9362bbf02a52 (127.27.40.131:34025)
I20260809 03:32:21.811584 27889 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:52105
I20260809 03:32:21.812937 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 03:32:21 dist-test-slave-gkw7 krb5kdc[27818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246341, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 03:32:21 dist-test-slave-gkw7 krb5kdc[27818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246341, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.40.129@KRBTEST.COM
I20260809 03:32:21.840101 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:21.831343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.27.40.129:42487)
0809 03:32:21.831565 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:21.831571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:21.831586 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:32:21.831620 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:21.831623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:21.831649 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:21.831742 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:21.832329 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.833199 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.834099 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.834510 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.834589 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:21.837227 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:21.837238 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:21.837239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:21.837247 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:21.839012 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:21.839389 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:21.839392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:21.839393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:21.839456 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:21.839631 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:21.839632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:21.839632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:21.839771 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:21.839894 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:21.839950 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 03:32:21.840016 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":124,"threads_started":1}
W20260809 03:32:21.840585 28040 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60646
Aug 09 03:32:21 dist-test-slave-gkw7 krb5kdc[27818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 03:32:21 dist-test-slave-gkw7 krb5kdc[27818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.129@KRBTEST.COM
I20260809 03:32:21.864641 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:21.857289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60658 (local address 127.27.40.129:42487)
0809 03:32:21.857378 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:21.857380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:21.857391 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:32:21.857423 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:21.857424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:21.857438 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:21.857496 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:21.858147 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.858745 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.859507 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:21.859726 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:21.859790 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:21.862103 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:21.862105 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:21.862105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:21.862113 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:21.863624 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:21.863936 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:21.863937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:21.863937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:21.863990 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:21.864153 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:21.864154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:21.864154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:21.864253 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:21.864346 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:21.864482 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 03:32:21.864544 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:32:21.865284 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 27974
I20260809 03:32:21.877812 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 28184
I20260809 03:32:21.892910 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 28377
I20260809 03:32:21.902567 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 27840
2026-08-09T03:32:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2213 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:32:21 dist-test-slave-gkw7 krb5kdc[28566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:32:21 dist-test-slave-gkw7 krb5kdc[28566](info): set up 2 sockets
Aug 09 03:32:21 dist-test-slave-gkw7 krb5kdc[28566](info): commencing operation
krb5kdc: starting...
W20260809 03:32:23.179445 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:32:23 dist-test-slave-gkw7 krb5kdc[28566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:32:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:32:23.300796 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:39275
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:39275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:23.382478 28585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:23.382716 28585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:23.382752 28585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:23.384429 28585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:23.384487 28585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:23.384510 28585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:23.384537 28585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:23.384565 28585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:32:23.386111 28585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:39275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:39275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:23.386389 28585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:23.386629 28585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:23.391541 28593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:23.391563 28585 server_base.cc:1034] running on GCE node
W20260809 03:32:23.391541 28595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:23.391577 28592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:23.392010 28585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:23.392304 28585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:23.393468 28585 hybrid_clock.cc:648] HybridClock initialized: now 1786246343393452 us; error 43 us; skew 500 ppm
Aug 09 03:32:23 dist-test-slave-gkw7 krb5kdc[28566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246343, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:23.396296 28585 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:32:23.396754 28585 webserver.cc:460] Webserver started at http://127.27.40.190:42605/ using document root <none> and password file <none>
I20260809 03:32:23.396996 28585 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:23.397063 28585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:23.397204 28585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:23.398309 28585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "22cebfb28bfb45259ca34a31c763fe4c"
format_stamp: "Formatted at 2026-08-09 03:32:23 on dist-test-slave-gkw7"
server_key: "acb3f48d1eaa166f88ddcf4e708a19ad"
server_key_iv: "2bd346a3293bbde9478d9d0eedb69e91"
server_key_version: "encryptionkey@0"
I20260809 03:32:23.398669 28585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "22cebfb28bfb45259ca34a31c763fe4c"
format_stamp: "Formatted at 2026-08-09 03:32:23 on dist-test-slave-gkw7"
server_key: "acb3f48d1eaa166f88ddcf4e708a19ad"
server_key_iv: "2bd346a3293bbde9478d9d0eedb69e91"
server_key_version: "encryptionkey@0"
I20260809 03:32:23.399982 28585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 03:32:23.400722 28608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:23.400925 28585 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 03:32:23.401028 28585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "22cebfb28bfb45259ca34a31c763fe4c"
format_stamp: "Formatted at 2026-08-09 03:32:23 on dist-test-slave-gkw7"
server_key: "acb3f48d1eaa166f88ddcf4e708a19ad"
server_key_iv: "2bd346a3293bbde9478d9d0eedb69e91"
server_key_version: "encryptionkey@0"
I20260809 03:32:23.401099 28585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:23.411940 28585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:23.413823 28585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:23.413991 28585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:23.418009 28585 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:39275
I20260809 03:32:23.418053 28675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:39275 every 8 connection(s)
I20260809 03:32:23.418413 28585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:23.419317 28676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:23.421834 28676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c: Bootstrap starting.
I20260809 03:32:23.422436 28676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:23.422766 28676 log.cc:824] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:23.423507 28676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c: No bootstrap required, opened a new log
I20260809 03:32:23.425102 28676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22cebfb28bfb45259ca34a31c763fe4c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 39275 } }
I20260809 03:32:23.425252 28676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:23.425312 28676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22cebfb28bfb45259ca34a31c763fe4c, State: Initialized, Role: FOLLOWER
I20260809 03:32:23.425484 28676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22cebfb28bfb45259ca34a31c763fe4c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 39275 } }
I20260809 03:32:23.425586 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 28585
I20260809 03:32:23.425612 28676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:23.425675 28676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:23.425678 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:23.425731 28676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:23.425848 27810 external_mini_cluster.cc:1428] Setting key 8699dea734803c45a2f7e5645aa03387
I20260809 03:32:23.426502 28676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22cebfb28bfb45259ca34a31c763fe4c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 39275 } }
I20260809 03:32:23.426640 28676 leader_election.cc:302] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22cebfb28bfb45259ca34a31c763fe4c; no voters:
I20260809 03:32:23.426852 28676 leader_election.cc:288] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:23.426874 28684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:23.427012 28684 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [term 1 LEADER]: Becoming Leader. State: Replica: 22cebfb28bfb45259ca34a31c763fe4c, State: Running, Role: LEADER
I20260809 03:32:23.427110 28684 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22cebfb28bfb45259ca34a31c763fe4c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 39275 } }
I20260809 03:32:23.427137 28676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:23.427433 28684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22cebfb28bfb45259ca34a31c763fe4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22cebfb28bfb45259ca34a31c763fe4c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 39275 } } }
I20260809 03:32:23.427516 28684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:23.427642 28684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22cebfb28bfb45259ca34a31c763fe4c. Latest consensus state: current_term: 1 leader_uuid: "22cebfb28bfb45259ca34a31c763fe4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22cebfb28bfb45259ca34a31c763fe4c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 39275 } } }
I20260809 03:32:23.427722 28684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:23.427934 28690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:23.428457 28690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:23.430035 28690 catalog_manager.cc:1347] Generated new cluster ID: 655379301c1c4cf9a4d1117b761e370a
I20260809 03:32:23.430087 28690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 03:32:23 dist-test-slave-gkw7 krb5kdc[28566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:23.438678 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:23.426426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38504 (local address 127.27.40.190:39275)
0809 03:32:23.426703 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:23.426712 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:23.426751 (+ 39us) server_negotiation.cc:407] Connection header received
0809 03:32:23.426898 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:23.426906 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:23.426967 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:23.427141 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:23.428092 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.429522 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.430622 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.431229 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.431318 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:23.434274 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:23.434278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:23.434278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:23.434290 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:23.436865 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:23.437332 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:23.437335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:23.437335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:23.437451 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:23.437710 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:23.437711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:23.437711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:23.438003 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:23.438155 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:23.438307 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 03:32:23.438439 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":113,"threads_started":1}
I20260809 03:32:23.439110 28690 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:23.439728 28690 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:23.442903 28690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22cebfb28bfb45259ca34a31c763fe4c: Generated new TSK 0
I20260809 03:32:23.443076 28690 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:32:23.473620 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:39275
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:23.557154 28712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:23.557401 28712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:23.557440 28712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:23.558676 28712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:23.558740 28712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:23.558786 28712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:32:23.560315 28712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:39275
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:23.560595 28712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:23.560825 28712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:23.566022 28719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:23.566030 28721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:23.566305 28712 server_base.cc:1034] running on GCE node
W20260809 03:32:23.566054 28723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:23.566583 28712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:23.566860 28712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:23.568035 28712 hybrid_clock.cc:648] HybridClock initialized: now 1786246343568011 us; error 40 us; skew 500 ppm
Aug 09 03:32:23 dist-test-slave-gkw7 krb5kdc[28566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246343, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:23.570817 28712 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:32:23.571305 28712 webserver.cc:460] Webserver started at http://127.27.40.129:46493/ using document root <none> and password file <none>
I20260809 03:32:23.571558 28712 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:23.571627 28712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:23.571781 28712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:23.572849 28712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "0e15e663f71b4613b9ffc433d953be09"
format_stamp: "Formatted at 2026-08-09 03:32:23 on dist-test-slave-gkw7"
server_key: "b9ce3157a8312034f3067bdad21cdece"
server_key_iv: "5fde654bc4a341119db2bacabb5224ba"
server_key_version: "encryptionkey@0"
I20260809 03:32:23.573230 28712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "0e15e663f71b4613b9ffc433d953be09"
format_stamp: "Formatted at 2026-08-09 03:32:23 on dist-test-slave-gkw7"
server_key: "b9ce3157a8312034f3067bdad21cdece"
server_key_iv: "5fde654bc4a341119db2bacabb5224ba"
server_key_version: "encryptionkey@0"
I20260809 03:32:23.574695 28712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:32:23.575524 28736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:23.575711 28712 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:23.575814 28712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "0e15e663f71b4613b9ffc433d953be09"
format_stamp: "Formatted at 2026-08-09 03:32:23 on dist-test-slave-gkw7"
server_key: "b9ce3157a8312034f3067bdad21cdece"
server_key_iv: "5fde654bc4a341119db2bacabb5224ba"
server_key_version: "encryptionkey@0"
I20260809 03:32:23.575897 28712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:23.586858 28712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:23.588891 28712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:23.589048 28712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:23.589274 28712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:23.589660 28712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:23.589715 28712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:23.589757 28712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:23.589784 28712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:23.595762 28712 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:35451
I20260809 03:32:23.595815 28907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:35451 every 8 connection(s)
I20260809 03:32:23.596156 28712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:32:23.598928 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 28712
I20260809 03:32:23.599037 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:32:23.599196 27810 external_mini_cluster.cc:1428] Setting key 93e41b7d821b0a1ed92c51f0f836f4e4
Aug 09 03:32:23 dist-test-slave-gkw7 krb5kdc[28566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246343, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:23.607362 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:23.596888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:37495 (local address 127.27.40.190:39275)
0809 03:32:23.596951 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:23.596953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:23.597201 (+ 248us) server_negotiation.cc:407] Connection header received
0809 03:32:23.598242 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:23.598243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:23.598262 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:23.598309 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:23.598947 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.599715 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.601219 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.601416 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.601472 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:23.603967 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:23.603968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:23.603969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:23.603975 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:23.605715 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:23.606229 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:23.606230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:23.606231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:23.606309 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:23.606617 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:23.606618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:23.606618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:23.606758 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:23.606841 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:23.607164 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 03:32:23.607259 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 03:32:23.607576 28911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:23.596858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:39275 (local address 127.27.40.129:37495)
0809 03:32:23.597056 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:23.597091 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:23.598118 (+ 1027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:23.598392 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:23.598396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:23.598648 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:23.598835 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:23.598838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.599825 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.599826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:23.601126 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:23.601131 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.601503 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.601505 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:23.601680 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:23.602289 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:23.602298 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:23.603878 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:23.605824 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:23.605832 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:23.605834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:23.606089 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:23.606389 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:23.606390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:23.606391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:23.606511 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:23.606843 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:23.606852 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:23.606996 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 03:32:23.607135 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 03:32:23.607311 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":90,"threads_started":1}
I20260809 03:32:23.608184 28908 heartbeater.cc:347] Connected to a master server at 127.27.40.190:39275
I20260809 03:32:23.608295 28908 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:23.608491 28908 heartbeater.cc:511] Master 127.27.40.190:39275 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
I20260809 03:32:23.609028 28629 ts_manager.cc:194] Registered new tserver with Master: 0e15e663f71b4613b9ffc433d953be09 (127.27.40.129:35451)
I20260809 03:32:23.609910 28629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:37495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:32:23.630697 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:39275
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:23.714392 28915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:23.714596 28915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:23.714619 28915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:23.715685 28915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:23.715719 28915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:23.715745 28915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:32:23.717264 28915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:39275
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:23.717463 28915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:23.717649 28915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:23.722626 28924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:23.722623 28923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:23.722656 28927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:23.722879 28915 server_base.cc:1034] running on GCE node
I20260809 03:32:23.723145 28915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:23.723351 28915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:23.724485 28915 hybrid_clock.cc:648] HybridClock initialized: now 1786246343724476 us; error 29 us; skew 500 ppm
Aug 09 03:32:23 dist-test-slave-gkw7 krb5kdc[28566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246343, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:23.727123 28915 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:32:23.727589 28915 webserver.cc:460] Webserver started at http://127.27.40.130:36701/ using document root <none> and password file <none>
I20260809 03:32:23.727831 28915 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:23.727876 28915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:23.728056 28915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:23.729210 28915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "f089c1a7fda049cba8edd7ebcc58c6f1"
format_stamp: "Formatted at 2026-08-09 03:32:23 on dist-test-slave-gkw7"
server_key: "3d76a8e135f59fff7442485cf337d8c6"
server_key_iv: "5a3be95d86d4c55b4692cf926e224e59"
server_key_version: "encryptionkey@0"
I20260809 03:32:23.729506 28915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "f089c1a7fda049cba8edd7ebcc58c6f1"
format_stamp: "Formatted at 2026-08-09 03:32:23 on dist-test-slave-gkw7"
server_key: "3d76a8e135f59fff7442485cf337d8c6"
server_key_iv: "5a3be95d86d4c55b4692cf926e224e59"
server_key_version: "encryptionkey@0"
I20260809 03:32:23.730872 28915 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:23.731549 28938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:23.731711 28915 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:23.731801 28915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "f089c1a7fda049cba8edd7ebcc58c6f1"
format_stamp: "Formatted at 2026-08-09 03:32:23 on dist-test-slave-gkw7"
server_key: "3d76a8e135f59fff7442485cf337d8c6"
server_key_iv: "5a3be95d86d4c55b4692cf926e224e59"
server_key_version: "encryptionkey@0"
I20260809 03:32:23.731854 28915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:23.749408 28915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:23.751814 28915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:23.751930 28915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:23.752111 28915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:23.752442 28915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:23.752480 28915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:23.752503 28915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:23.752550 28915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:23.758224 28915 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:32983
I20260809 03:32:23.758315 29105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:32983 every 8 connection(s)
I20260809 03:32:23.758638 28915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
Aug 09 03:32:23 dist-test-slave-gkw7 krb5kdc[28566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246343, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:23.766708 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 28915
I20260809 03:32:23.766813 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:32:23.767001 27810 external_mini_cluster.cc:1428] Setting key 175c82cb1fdfb5d55e686276d91df2ec
I20260809 03:32:23.768428 29108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:23.759385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:39275 (local address 127.27.40.130:51913)
0809 03:32:23.759567 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:23.759614 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:23.760592 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:23.760874 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:23.760878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:23.761165 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:23.761358 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:23.761362 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.762145 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.762146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:23.762974 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:23.762976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.763379 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.763381 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:23.763509 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:23.764050 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:23.764058 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:23.765415 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:23.766879 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:23.766886 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:23.766888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:23.767188 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:23.767450 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:23.767451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:23.767451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:23.767587 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:23.767930 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:23.767935 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:23.768055 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 03:32:23.768146 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 03:32:23.768269 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260809 03:32:23.768431 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:23.759423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:51913 (local address 127.27.40.190:39275)
0809 03:32:23.759507 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:23.759509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:23.759700 (+ 191us) server_negotiation.cc:407] Connection header received
0809 03:32:23.760733 (+ 1033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:23.760735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:23.760751 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:23.760803 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:23.761460 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.762045 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.763075 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.763283 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.763369 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:23.765513 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:23.765515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:23.765516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:23.765524 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:23.766772 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:23.767294 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:23.767295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:23.767296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:23.767369 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:23.767694 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:23.767694 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:23.767695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:23.767860 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:23.767936 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:23.768192 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 03:32:23.768290 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 03:32:23.769019 29106 heartbeater.cc:347] Connected to a master server at 127.27.40.190:39275
I20260809 03:32:23.769132 29106 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:23.769325 29106 heartbeater.cc:511] Master 127.27.40.190:39275 requested a full tablet report, sending...
I20260809 03:32:23.769781 28629 ts_manager.cc:194] Registered new tserver with Master: f089c1a7fda049cba8edd7ebcc58c6f1 (127.27.40.130:32983)
I20260809 03:32:23.770190 28629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:51913
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:32:23.792528 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:39275
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:23.876556 29112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:23.876761 29112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:23.876786 29112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:23.878058 29112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:23.878100 29112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:23.878125 29112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:32:23.879733 29112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:39275
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:23.879992 29112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:23.880240 29112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:23.885409 29122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:23.885421 29128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:23.885428 29121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:23.885699 29112 server_base.cc:1034] running on GCE node
I20260809 03:32:23.885890 29112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:23.886083 29112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:23.887264 29112 hybrid_clock.cc:648] HybridClock initialized: now 1786246343887229 us; error 44 us; skew 500 ppm
Aug 09 03:32:23 dist-test-slave-gkw7 krb5kdc[28566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246343, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:23.889890 29112 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:32:23.890372 29112 webserver.cc:460] Webserver started at http://127.27.40.131:43911/ using document root <none> and password file <none>
I20260809 03:32:23.890599 29112 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:23.890643 29112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:23.890817 29112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:23.891865 29112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "e396b4d237814dd796afd62d8e0b16b9"
format_stamp: "Formatted at 2026-08-09 03:32:23 on dist-test-slave-gkw7"
server_key: "544d935b0e91be9dee92e8ebe911fb1f"
server_key_iv: "5dae8e87e2c58fd235451d6d069e2284"
server_key_version: "encryptionkey@0"
I20260809 03:32:23.892230 29112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "e396b4d237814dd796afd62d8e0b16b9"
format_stamp: "Formatted at 2026-08-09 03:32:23 on dist-test-slave-gkw7"
server_key: "544d935b0e91be9dee92e8ebe911fb1f"
server_key_iv: "5dae8e87e2c58fd235451d6d069e2284"
server_key_version: "encryptionkey@0"
I20260809 03:32:23.893675 29112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 03:32:23.894457 29139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:23.894651 29112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:23.894745 29112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "e396b4d237814dd796afd62d8e0b16b9"
format_stamp: "Formatted at 2026-08-09 03:32:23 on dist-test-slave-gkw7"
server_key: "544d935b0e91be9dee92e8ebe911fb1f"
server_key_iv: "5dae8e87e2c58fd235451d6d069e2284"
server_key_version: "encryptionkey@0"
I20260809 03:32:23.894819 29112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:23.907135 29112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:23.909432 29112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:23.909621 29112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:23.909871 29112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:23.910183 29112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:23.910219 29112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:23.910262 29112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:23.910290 29112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:23.919010 29112 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:46715
I20260809 03:32:23.919108 29312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:46715 every 8 connection(s)
I20260809 03:32:23.919463 29112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
Aug 09 03:32:23 dist-test-slave-gkw7 krb5kdc[28566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246343, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:23.928005 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 29112
I20260809 03:32:23.928140 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:32:23.928310 27810 external_mini_cluster.cc:1428] Setting key 7e67b97124bb94b7c4b8c2c1c33bd135
I20260809 03:32:23.929834 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:23.920327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:50777 (local address 127.27.40.190:39275)
0809 03:32:23.920413 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:23.920415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:23.920673 (+ 258us) server_negotiation.cc:407] Connection header received
0809 03:32:23.921827 (+ 1154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:23.921829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:23.921840 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:23.921898 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:23.922547 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.923051 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.924045 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.924235 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.924289 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:23.926474 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:23.926475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:23.926475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:23.926483 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:23.928191 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:23.928699 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:23.928701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:23.928701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:23.928766 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:23.929041 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:23.929042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:23.929042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:23.929236 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:23.929330 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:23.929657 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 03:32:23.929741 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:32:23.930027 29316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:23.920292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:39275 (local address 127.27.40.131:50777)
0809 03:32:23.920474 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:23.920511 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:23.921723 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:23.921954 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:23.921957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:23.922260 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:23.922458 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:23.922462 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.923136 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.923137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:23.923972 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:23.923974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.924307 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.924308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:23.924410 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:23.924893 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:23.924901 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:23.926378 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:23.928284 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:23.928291 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:23.928294 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:23.928628 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:23.928833 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:23.928834 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:23.928835 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:23.928976 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:23.929322 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:23.929329 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:23.929467 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 03:32:23.929622 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 03:32:23.929787 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":98,"threads_started":1}
I20260809 03:32:23.930534 29313 heartbeater.cc:347] Connected to a master server at 127.27.40.190:39275
I20260809 03:32:23.930639 29313 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:23.930828 29313 heartbeater.cc:511] Master 127.27.40.190:39275 requested a full tablet report, sending...
I20260809 03:32:23.931209 28629 ts_manager.cc:194] Registered new tserver with Master: e396b4d237814dd796afd62d8e0b16b9 (127.27.40.131:46715)
I20260809 03:32:23.931550 28629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:50777
I20260809 03:32:23.940269 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:32:23.948176 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:23.941997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38518 (local address 127.27.40.190:39275)
0809 03:32:23.942062 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:23.942064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:23.942184 (+ 120us) server_negotiation.cc:407] Connection header received
0809 03:32:23.942224 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:23.942225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:23.942235 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:23.942293 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:23.942839 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.943375 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.944109 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.944309 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.944385 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:23.945612 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:23.945613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:23.945614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:23.945621 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:23.947003 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:23.947354 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:23.947355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:23.947356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:23.947419 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:23.947621 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:23.947622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:23.947623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:23.947774 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:23.947885 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:23.947993 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 03:32:23.948073 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 03:32:23.950347 28629 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:32:23.952010 28629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:23.959195 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:23.954888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46746 (local address 127.27.40.131:46715)
0809 03:32:23.955053 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:23.955057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:23.955656 (+ 599us) server_negotiation.cc:407] Connection header received
0809 03:32:23.955707 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:23.955709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:23.955725 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:23.955809 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:23.955961 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.956952 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.958129 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.958743 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.958817 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:23.958872 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:23.959052 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 03:32:23.959095 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260809 03:32:23.959197 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:23.954670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:46715 (local address 127.0.0.1:46746)
0809 03:32:23.955545 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:23.955555 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:23.955642 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:23.955830 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:23.955831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:23.955834 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:23.955892 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:23.955894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.957099 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.957101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:23.958032 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:23.958035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.958838 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.958839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:23.958929 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:23.958944 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 03:32:23.959028 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 03:32:23.959092 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":95,"threads_started":1}
I20260809 03:32:23.959609 29330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:23.954455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:32983 (local address 127.0.0.1:44318)
0809 03:32:23.954727 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:23.954770 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:23.954845 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:23.955304 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:23.955307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:23.955310 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:23.955421 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:23.955423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.956855 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.956856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:23.958346 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:23.958350 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.959398 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.959400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:23.959463 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:23.959467 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:23.959516 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 03:32:23.959561 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":81,"threads_started":1}
I20260809 03:32:23.959908 29336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:23.954719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44318 (local address 127.27.40.130:32983)
0809 03:32:23.955077 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:23.955081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:23.955091 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:32:23.955143 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:23.955145 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:23.955159 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:23.955295 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:23.955567 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.956732 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.958453 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.959132 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.959200 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:23.959254 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:23.959786 (+ 532us) server_negotiation.cc:299] Negotiation successful
0809 03:32:23.959826 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":84,"threads_started":1}
I20260809 03:32:23.960074 29328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:23.954455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:35451 (local address 127.0.0.1:59030)
0809 03:32:23.954774 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:23.954784 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:23.954845 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:23.956049 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:23.956050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:23.956052 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:23.956099 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:23.956110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.957416 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.957417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:23.958502 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:23.958504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.959853 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.959854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:23.959919 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:23.959923 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:23.959959 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 03:32:23.959998 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":2,"thread_start_us":55,"threads_started":1}
I20260809 03:32:23.960108 29216 tablet_service.cc:1467] Processing CreateTablet for tablet 073a3bef9ba04a409b3d52a6fb364747 (DEFAULT_TABLE table=test-table [id=22b5b881216f4a4b9cc20d39d0da44c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:23.960520 29216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 073a3bef9ba04a409b3d52a6fb364747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:23.960518 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:23.954603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59030 (local address 127.27.40.129:35451)
0809 03:32:23.955856 (+ 1253us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:23.955861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:23.955872 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:32:23.955934 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:23.955937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:23.955955 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:23.956046 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:23.956272 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.957191 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.958604 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:23.959734 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:23.960287 (+ 553us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:23.960351 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:23.960379 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 03:32:23.960422 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1191,"thread_start_us":85,"threads_started":1}
I20260809 03:32:23.960503 29009 tablet_service.cc:1467] Processing CreateTablet for tablet 073a3bef9ba04a409b3d52a6fb364747 (DEFAULT_TABLE table=test-table [id=22b5b881216f4a4b9cc20d39d0da44c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:23.960835 29009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 073a3bef9ba04a409b3d52a6fb364747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:23.961390 28811 tablet_service.cc:1467] Processing CreateTablet for tablet 073a3bef9ba04a409b3d52a6fb364747 (DEFAULT_TABLE table=test-table [id=22b5b881216f4a4b9cc20d39d0da44c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:23.961767 28811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 073a3bef9ba04a409b3d52a6fb364747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:23.964076 29340 tablet_bootstrap.cc:492] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1: Bootstrap starting.
I20260809 03:32:23.964804 29340 tablet_bootstrap.cc:654] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:23.964998 29341 tablet_bootstrap.cc:492] T 073a3bef9ba04a409b3d52a6fb364747 P e396b4d237814dd796afd62d8e0b16b9: Bootstrap starting.
I20260809 03:32:23.965647 29340 log.cc:824] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:23.965888 29341 tablet_bootstrap.cc:654] T 073a3bef9ba04a409b3d52a6fb364747 P e396b4d237814dd796afd62d8e0b16b9: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:23.966138 29342 tablet_bootstrap.cc:492] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09: Bootstrap starting.
I20260809 03:32:23.966492 29341 log.cc:824] T 073a3bef9ba04a409b3d52a6fb364747 P e396b4d237814dd796afd62d8e0b16b9: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:23.966810 29342 tablet_bootstrap.cc:654] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:23.967198 29340 tablet_bootstrap.cc:492] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1: No bootstrap required, opened a new log
I20260809 03:32:23.967252 29340 ts_tablet_manager.cc:1397] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 03:32:23.967478 29342 log.cc:824] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:23.968228 29342 tablet_bootstrap.cc:492] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09: No bootstrap required, opened a new log
I20260809 03:32:23.968288 29342 ts_tablet_manager.cc:1397] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:32:23.968914 29340 raft_consensus.cc:348] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32983 } } peers { permanent_uuid: "0e15e663f71b4613b9ffc433d953be09" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 35451 } } peers { permanent_uuid: "e396b4d237814dd796afd62d8e0b16b9" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46715 } }
I20260809 03:32:23.969091 29340 raft_consensus.cc:374] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:23.969139 29340 raft_consensus.cc:729] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f089c1a7fda049cba8edd7ebcc58c6f1, State: Initialized, Role: FOLLOWER
I20260809 03:32:23.969272 29340 consensus_queue.cc:260] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32983 } } peers { permanent_uuid: "0e15e663f71b4613b9ffc433d953be09" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 35451 } } peers { permanent_uuid: "e396b4d237814dd796afd62d8e0b16b9" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46715 } }
I20260809 03:32:23.969609 29106 heartbeater.cc:503] Master 127.27.40.190:39275 was elected leader, sending a full tablet report...
I20260809 03:32:23.969923 29340 ts_tablet_manager.cc:1428] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 03:32:23.969946 29341 tablet_bootstrap.cc:492] T 073a3bef9ba04a409b3d52a6fb364747 P e396b4d237814dd796afd62d8e0b16b9: No bootstrap required, opened a new log
I20260809 03:32:23.970016 29341 ts_tablet_manager.cc:1397] T 073a3bef9ba04a409b3d52a6fb364747 P e396b4d237814dd796afd62d8e0b16b9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 03:32:23.970428 29342 raft_consensus.cc:348] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32983 } } peers { permanent_uuid: "0e15e663f71b4613b9ffc433d953be09" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 35451 } } peers { permanent_uuid: "e396b4d237814dd796afd62d8e0b16b9" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46715 } }
I20260809 03:32:23.970633 29342 raft_consensus.cc:374] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:23.970680 29342 raft_consensus.cc:729] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e15e663f71b4613b9ffc433d953be09, State: Initialized, Role: FOLLOWER
I20260809 03:32:23.970799 29342 consensus_queue.cc:260] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32983 } } peers { permanent_uuid: "0e15e663f71b4613b9ffc433d953be09" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 35451 } } peers { permanent_uuid: "e396b4d237814dd796afd62d8e0b16b9" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46715 } }
I20260809 03:32:23.971088 29342 ts_tablet_manager.cc:1428] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:32:23.971109 28908 heartbeater.cc:503] Master 127.27.40.190:39275 was elected leader, sending a full tablet report...
I20260809 03:32:23.972215 29341 raft_consensus.cc:348] T 073a3bef9ba04a409b3d52a6fb364747 P e396b4d237814dd796afd62d8e0b16b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32983 } } peers { permanent_uuid: "0e15e663f71b4613b9ffc433d953be09" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 35451 } } peers { permanent_uuid: "e396b4d237814dd796afd62d8e0b16b9" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46715 } }
I20260809 03:32:23.972383 29341 raft_consensus.cc:374] T 073a3bef9ba04a409b3d52a6fb364747 P e396b4d237814dd796afd62d8e0b16b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:23.972434 29341 raft_consensus.cc:729] T 073a3bef9ba04a409b3d52a6fb364747 P e396b4d237814dd796afd62d8e0b16b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e396b4d237814dd796afd62d8e0b16b9, State: Initialized, Role: FOLLOWER
I20260809 03:32:23.972581 29341 consensus_queue.cc:260] T 073a3bef9ba04a409b3d52a6fb364747 P e396b4d237814dd796afd62d8e0b16b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32983 } } peers { permanent_uuid: "0e15e663f71b4613b9ffc433d953be09" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 35451 } } peers { permanent_uuid: "e396b4d237814dd796afd62d8e0b16b9" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46715 } }
I20260809 03:32:23.972918 29341 ts_tablet_manager.cc:1428] T 073a3bef9ba04a409b3d52a6fb364747 P e396b4d237814dd796afd62d8e0b16b9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:32:23.972918 29313 heartbeater.cc:503] Master 127.27.40.190:39275 was elected leader, sending a full tablet report...
W20260809 03:32:24.009218 29107 tablet.cc:2369] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:32:24.097010 28909 tablet.cc:2369] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:32:24.139434 29351 raft_consensus.cc:482] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:24.139557 29351 raft_consensus.cc:504] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32983 } } peers { permanent_uuid: "0e15e663f71b4613b9ffc433d953be09" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 35451 } } peers { permanent_uuid: "e396b4d237814dd796afd62d8e0b16b9" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46715 } }
I20260809 03:32:24.139909 29352 raft_consensus.cc:482] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:24.140020 29351 leader_election.cc:288] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e15e663f71b4613b9ffc433d953be09 (127.27.40.129:35451), e396b4d237814dd796afd62d8e0b16b9 (127.27.40.131:46715)
I20260809 03:32:24.140038 29352 raft_consensus.cc:504] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32983 } } peers { permanent_uuid: "0e15e663f71b4613b9ffc433d953be09" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 35451 } } peers { permanent_uuid: "e396b4d237814dd796afd62d8e0b16b9" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46715 } }
I20260809 03:32:24.140867 29352 leader_election.cc:288] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f089c1a7fda049cba8edd7ebcc58c6f1 (127.27.40.130:32983), e396b4d237814dd796afd62d8e0b16b9 (127.27.40.131:46715)
I20260809 03:32:24.144088 29108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:24.140212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:35451 (local address 127.27.40.130:60139)
0809 03:32:24.140281 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:24.140298 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:24.140400 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:24.140925 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:24.140926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:24.140928 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:24.140972 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:24.140973 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.141915 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.141916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:24.143109 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:24.143112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.143844 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.143845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:24.143918 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:24.143922 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:24.143971 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 03:32:24.144012 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 03:32:24.144376 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:24.140634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:60139 (local address 127.27.40.129:35451)
0809 03:32:24.140678 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:24.140680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:24.140689 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:24.140714 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:24.140715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:24.140725 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:24.140795 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:24.141034 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.141794 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.143227 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.143742 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.143805 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:24.143813 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:24.144281 (+ 468us) server_negotiation.cc:299] Negotiation successful
0809 03:32:24.144317 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 03:32:24.144582 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:24.140575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:50535 (local address 127.27.40.131:46715)
0809 03:32:24.140657 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:24.140659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:24.140668 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:24.140696 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:24.140697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:24.140706 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:24.140759 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:24.141047 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.141852 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.143406 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.144038 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.144085 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:24.144091 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:24.144467 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 03:32:24.144503 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:32:24.144721 28839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "073a3bef9ba04a409b3d52a6fb364747" candidate_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e15e663f71b4613b9ffc433d953be09" is_pre_election: true
I20260809 03:32:24.144794 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:24.140330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:46715 (local address 127.27.40.130:50535)
0809 03:32:24.140562 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:24.140572 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:24.140616 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:24.140866 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:24.140867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:24.140869 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:24.140951 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:24.140954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.141980 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.141981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:24.143303 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:24.143306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.144290 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.144292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:24.144370 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:24.144373 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:32:24.144669 (+ 296us) client_negotiation.cc:240] Negotiation successful
0809 03:32:24.144717 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":97,"threads_started":1}
I20260809 03:32:24.144831 28839 raft_consensus.cc:2454] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f089c1a7fda049cba8edd7ebcc58c6f1 in term 0.
I20260809 03:32:24.145059 29358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:24.140758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:46715 (local address 127.27.40.129:34611)
0809 03:32:24.140973 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:24.140982 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:24.141027 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:24.141233 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:24.141234 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:24.141236 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:24.141299 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:24.141301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.142320 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.142321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:24.143562 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:24.143565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.144730 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.144731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:24.144830 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:24.144905 (+ 75us) client_negotiation.cc:769] Sending connection context
0809 03:32:24.144957 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 03:32:24.145003 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":67,"threads_started":1}
I20260809 03:32:24.145045 28941 leader_election.cc:302] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e15e663f71b4613b9ffc433d953be09, f089c1a7fda049cba8edd7ebcc58c6f1; no voters:
I20260809 03:32:24.145229 29351 raft_consensus.cc:2781] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:24.145285 29336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:24.140588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:46401 (local address 127.27.40.130:32983)
0809 03:32:24.140816 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:24.140819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:24.140930 (+ 111us) server_negotiation.cc:407] Connection header received
0809 03:32:24.141636 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:24.141637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:24.141657 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:24.141688 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:24.141924 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.142609 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.143995 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.144563 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.144676 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:24.144685 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:24.145140 (+ 455us) server_negotiation.cc:299] Negotiation successful
0809 03:32:24.145198 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209}
I20260809 03:32:24.145305 29351 raft_consensus.cc:482] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:24.145331 29351 raft_consensus.cc:3037] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:24.145440 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:24.140781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:34611 (local address 127.27.40.131:46715)
0809 03:32:24.141107 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:24.141109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:24.141116 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:32:24.141133 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:24.141133 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:24.141141 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:24.141213 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:24.141387 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.142196 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.143652 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.144544 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.144604 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:24.144612 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:24.145216 (+ 604us) server_negotiation.cc:299] Negotiation successful
0809 03:32:24.145259 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":111,"threads_started":1}
I20260809 03:32:24.145465 29357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:24.140507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:32983 (local address 127.27.40.129:46401)
0809 03:32:24.140869 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:24.140879 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:24.140958 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:24.141746 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:24.141747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:24.141749 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:24.141829 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:24.141831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.142721 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.142721 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:24.143905 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:24.143908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.144981 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.144982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:24.145045 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:24.145050 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:32:24.145115 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 03:32:24.145269 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":51,"threads_started":1}
I20260809 03:32:24.145792 29243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "073a3bef9ba04a409b3d52a6fb364747" candidate_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e396b4d237814dd796afd62d8e0b16b9" is_pre_election: true
I20260809 03:32:24.145872 29242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "073a3bef9ba04a409b3d52a6fb364747" candidate_uuid: "0e15e663f71b4613b9ffc433d953be09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e396b4d237814dd796afd62d8e0b16b9" is_pre_election: true
I20260809 03:32:24.145932 29243 raft_consensus.cc:2454] T 073a3bef9ba04a409b3d52a6fb364747 P e396b4d237814dd796afd62d8e0b16b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f089c1a7fda049cba8edd7ebcc58c6f1 in term 0.
I20260809 03:32:24.145907 29038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "073a3bef9ba04a409b3d52a6fb364747" candidate_uuid: "0e15e663f71b4613b9ffc433d953be09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" is_pre_election: true
I20260809 03:32:24.146374 29351 raft_consensus.cc:504] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32983 } } peers { permanent_uuid: "0e15e663f71b4613b9ffc433d953be09" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 35451 } } peers { permanent_uuid: "e396b4d237814dd796afd62d8e0b16b9" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46715 } }
I20260809 03:32:24.146500 29038 raft_consensus.cc:2379] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e15e663f71b4613b9ffc433d953be09 in current term 1: Already voted for candidate f089c1a7fda049cba8edd7ebcc58c6f1 in this term.
I20260809 03:32:24.146521 29351 leader_election.cc:288] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [CANDIDATE]: Term 1 election: Requested vote from peers 0e15e663f71b4613b9ffc433d953be09 (127.27.40.129:35451), e396b4d237814dd796afd62d8e0b16b9 (127.27.40.131:46715)
I20260809 03:32:24.146773 28839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "073a3bef9ba04a409b3d52a6fb364747" candidate_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e15e663f71b4613b9ffc433d953be09"
I20260809 03:32:24.146776 29242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "073a3bef9ba04a409b3d52a6fb364747" candidate_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e396b4d237814dd796afd62d8e0b16b9"
I20260809 03:32:24.146862 28839 raft_consensus.cc:3037] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:24.146862 29242 raft_consensus.cc:3037] T 073a3bef9ba04a409b3d52a6fb364747 P e396b4d237814dd796afd62d8e0b16b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:24.147866 28839 raft_consensus.cc:2454] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f089c1a7fda049cba8edd7ebcc58c6f1 in term 1.
I20260809 03:32:24.147918 29242 raft_consensus.cc:2454] T 073a3bef9ba04a409b3d52a6fb364747 P e396b4d237814dd796afd62d8e0b16b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f089c1a7fda049cba8edd7ebcc58c6f1 in term 1.
I20260809 03:32:24.148106 28941 leader_election.cc:302] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e396b4d237814dd796afd62d8e0b16b9, f089c1a7fda049cba8edd7ebcc58c6f1; no voters:
I20260809 03:32:24.148114 28739 leader_election.cc:302] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e15e663f71b4613b9ffc433d953be09; no voters: e396b4d237814dd796afd62d8e0b16b9, f089c1a7fda049cba8edd7ebcc58c6f1
I20260809 03:32:24.148229 29351 raft_consensus.cc:2781] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:24.148248 29352 raft_consensus.cc:2726] T 073a3bef9ba04a409b3d52a6fb364747 P 0e15e663f71b4613b9ffc433d953be09 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 03:32:24.148402 29351 raft_consensus.cc:686] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [term 1 LEADER]: Becoming Leader. State: Replica: f089c1a7fda049cba8edd7ebcc58c6f1, State: Running, Role: LEADER
I20260809 03:32:24.148530 29351 consensus_queue.cc:237] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32983 } } peers { permanent_uuid: "0e15e663f71b4613b9ffc433d953be09" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 35451 } } peers { permanent_uuid: "e396b4d237814dd796afd62d8e0b16b9" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46715 } }
I20260809 03:32:24.149338 28628 catalog_manager.cc:5495] T 073a3bef9ba04a409b3d52a6fb364747 P f089c1a7fda049cba8edd7ebcc58c6f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to f089c1a7fda049cba8edd7ebcc58c6f1 (127.27.40.130). New cstate: current_term: 1 leader_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f089c1a7fda049cba8edd7ebcc58c6f1" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e15e663f71b4613b9ffc433d953be09" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 35451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e396b4d237814dd796afd62d8e0b16b9" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46715 } health_report { overall_health: UNKNOWN } } }
W20260809 03:32:24.170207 29315 tablet.cc:2369] T 073a3bef9ba04a409b3d52a6fb364747 P e396b4d237814dd796afd62d8e0b16b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 03:32:24 dist-test-slave-gkw7 krb5kdc[28566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.129@KRBTEST.COM
I20260809 03:32:24.192246 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:24.184563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59042 (local address 127.27.40.129:35451)
0809 03:32:24.184664 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:24.184667 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:24.184719 (+ 52us) server_negotiation.cc:407] Connection header received
0809 03:32:24.184768 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:24.184769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:24.184779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:24.184833 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:24.185511 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.186071 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.186831 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.187020 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.187081 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:24.189465 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:24.189478 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:24.189478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:24.189484 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:24.191256 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:24.191557 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:24.191559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:24.191560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:24.191632 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:24.191786 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:24.191787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:24.191787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:24.191910 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:24.192008 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:24.192063 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 03:32:24.192128 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 03:32:24 dist-test-slave-gkw7 krb5kdc[28566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246344, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 03:32:24 dist-test-slave-gkw7 krb5kdc[28566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246344, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.40.129@KRBTEST.COM
I20260809 03:32:24.217518 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:24.210024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59054 (local address 127.27.40.129:35451)
0809 03:32:24.210102 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:24.210105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:24.210207 (+ 102us) server_negotiation.cc:407] Connection header received
0809 03:32:24.210285 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:24.210286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:24.210302 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:24.210369 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:24.210939 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.211423 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.212151 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.212353 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.212430 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:24.215116 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:24.215117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:24.215118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:24.215126 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:24.216316 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:24.216647 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:24.216649 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:24.216649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:24.216710 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:24.216925 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:24.216926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:24.216927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:24.217027 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:24.217132 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:24.217328 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 03:32:24.217404 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 03:32:24 dist-test-slave-gkw7 krb5kdc[28566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 03:32:24 dist-test-slave-gkw7 krb5kdc[28566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.129@KRBTEST.COM
I20260809 03:32:24.241258 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:24.234279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59070 (local address 127.27.40.129:35451)
0809 03:32:24.234366 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:24.234368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:24.234381 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:32:24.234460 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:24.234461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:24.234472 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:24.234575 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:24.235108 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.235707 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.236460 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:24.236659 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:24.236719 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:24.238883 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:24.238884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:24.238884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:24.238891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:24.240129 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:24.240445 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:24.240446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:24.240446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:24.240532 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:24.240770 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:24.240771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:24.240771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:24.240880 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:24.240971 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:24.241108 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 03:32:24.241177 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 03:32:24.241789 28779 tablet_service.cc:2908] Rejecting scan request for tablet 073a3bef9ba04a409b3d52a6fb364747: Uninitialized: clean time has not yet been initialized
I20260809 03:32:24.242784 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 28712
I20260809 03:32:24.258523 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 28915
I20260809 03:32:24.266326 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 29112
I20260809 03:32:24.277195 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 28585
2026-08-09T03:32:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2370 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:32:24 dist-test-slave-gkw7 krb5kdc[29383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:32:24 dist-test-slave-gkw7 krb5kdc[29383](info): set up 2 sockets
Aug 09 03:32:24 dist-test-slave-gkw7 krb5kdc[29383](info): commencing operation
krb5kdc: starting...
W20260809 03:32:25.520186 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:32:25 dist-test-slave-gkw7 krb5kdc[29383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:32:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:32:25.645849 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:41295
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:46531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:41295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:25.728545 29412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:25.728776 29412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:25.728813 29412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:25.729967 29412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:25.730032 29412 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 03:32:25.730064 29412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:25.730095 29412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:25.730122 29412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:25.730146 29412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:32:25.731564 29412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:46531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:41295
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:41295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:25.731830 29412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:25.732043 29412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:25.736984 29423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:25.737048 29412 server_base.cc:1034] running on GCE node
W20260809 03:32:25.736996 29418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:25.736994 29420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:25.737457 29412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:25.737821 29412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:25.739101 29412 hybrid_clock.cc:648] HybridClock initialized: now 1786246345739093 us; error 35 us; skew 500 ppm
Aug 09 03:32:25 dist-test-slave-gkw7 krb5kdc[29383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246345, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:25.750859 29412 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:32:25.751320 29412 webserver.cc:460] Webserver started at http://127.27.40.190:33113/ using document root <none> and password file <none>
I20260809 03:32:25.751559 29412 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:25.751628 29412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:25.751787 29412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:25.752875 29412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "f07288df997d4a6f822312d866cae7ae"
format_stamp: "Formatted at 2026-08-09 03:32:25 on dist-test-slave-gkw7"
server_key: "5a131922d76649dc533cf2382ad7912b"
server_key_iv: "d20350dcc7b6622ccb166c9e605b8028"
server_key_version: "encryptionkey@0"
I20260809 03:32:25.753248 29412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "f07288df997d4a6f822312d866cae7ae"
format_stamp: "Formatted at 2026-08-09 03:32:25 on dist-test-slave-gkw7"
server_key: "5a131922d76649dc533cf2382ad7912b"
server_key_iv: "d20350dcc7b6622ccb166c9e605b8028"
server_key_version: "encryptionkey@0"
I20260809 03:32:25.754721 29412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:32:25.755505 29431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:25.755681 29412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:25.755787 29412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "f07288df997d4a6f822312d866cae7ae"
format_stamp: "Formatted at 2026-08-09 03:32:25 on dist-test-slave-gkw7"
server_key: "5a131922d76649dc533cf2382ad7912b"
server_key_iv: "d20350dcc7b6622ccb166c9e605b8028"
server_key_version: "encryptionkey@0"
I20260809 03:32:25.755872 29412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:25.771917 29412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:25.773901 29412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:25.774081 29412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:25.778887 29412 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:41295
I20260809 03:32:25.778956 29526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:41295 every 8 connection(s)
I20260809 03:32:25.779268 29412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:25.779942 29527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:25.780752 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 29412
I20260809 03:32:25.780851 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:25.781015 27810 external_mini_cluster.cc:1428] Setting key 70393308fd4c63f67916d81200fdbb01
I20260809 03:32:25.782881 29527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae: Bootstrap starting.
I20260809 03:32:25.783470 29527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:25.783785 29527 log.cc:824] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:25.784466 29527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae: No bootstrap required, opened a new log
I20260809 03:32:25.786497 29527 raft_consensus.cc:348] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f07288df997d4a6f822312d866cae7ae" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41295 } }
I20260809 03:32:25.786664 29527 raft_consensus.cc:374] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:25.786744 29527 raft_consensus.cc:729] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f07288df997d4a6f822312d866cae7ae, State: Initialized, Role: FOLLOWER
I20260809 03:32:25.786898 29527 consensus_queue.cc:260] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f07288df997d4a6f822312d866cae7ae" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41295 } }
I20260809 03:32:25.786986 29527 raft_consensus.cc:388] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:25.787027 29527 raft_consensus.cc:482] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:25.787075 29527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [term 0 FOLLOWER]: Advancing to term 1
Aug 09 03:32:25 dist-test-slave-gkw7 krb5kdc[29383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:25.787645 29527 raft_consensus.cc:504] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f07288df997d4a6f822312d866cae7ae" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41295 } }
I20260809 03:32:25.787763 29527 leader_election.cc:302] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f07288df997d4a6f822312d866cae7ae; no voters:
I20260809 03:32:25.787988 29527 leader_election.cc:288] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:25.788015 29533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:25.788139 29533 raft_consensus.cc:686] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [term 1 LEADER]: Becoming Leader. State: Replica: f07288df997d4a6f822312d866cae7ae, State: Running, Role: LEADER
I20260809 03:32:25.788223 29533 consensus_queue.cc:237] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f07288df997d4a6f822312d866cae7ae" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41295 } }
I20260809 03:32:25.788252 29527 sys_catalog.cc:564] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:25.788537 29533 sys_catalog.cc:455] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader f07288df997d4a6f822312d866cae7ae. Latest consensus state: current_term: 1 leader_uuid: "f07288df997d4a6f822312d866cae7ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f07288df997d4a6f822312d866cae7ae" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41295 } } }
I20260809 03:32:25.788548 29535 sys_catalog.cc:455] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f07288df997d4a6f822312d866cae7ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f07288df997d4a6f822312d866cae7ae" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41295 } } }
I20260809 03:32:25.788622 29533 sys_catalog.cc:458] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:25.788686 29535 sys_catalog.cc:458] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:25.789108 29541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:25.789727 29541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:25.791496 29541 catalog_manager.cc:1347] Generated new cluster ID: f0e8cc07fdb349bdb463c9ae9fb4d951
I20260809 03:32:25.791549 29541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:32:25.792407 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:25.781571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37570 (local address 127.27.40.190:41295)
0809 03:32:25.781763 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:25.781779 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:25.781813 (+ 34us) server_negotiation.cc:407] Connection header received
0809 03:32:25.782234 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:25.782240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:25.782286 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:25.782433 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:25.783102 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:25.784117 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:25.784907 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:25.785311 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:25.785427 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:25.787887 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:25.787892 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:25.787892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:25.787910 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:25.790265 (+ 2355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:25.790668 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:25.790671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:25.790672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:25.790774 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:25.791156 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:25.791157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:25.791157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:25.791440 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:25.791997 (+ 557us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:25.792081 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 03:32:25.792211 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":79,"threads_started":1}
I20260809 03:32:25.811700 29541 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:25.812460 29541 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:25.821228 29541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae: Generated new TSK 0
I20260809 03:32:25.821408 29541 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:32:25.852744 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:41295
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:46531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:25.934576 29563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:25.934801 29563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:25.934842 29563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:25.935994 29563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:25.936061 29563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:25.936100 29563 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:32:25.936156 29563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:32:25.937664 29563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:46531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.40.190:41295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:25.937927 29563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:25.938158 29563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:25.943830 29573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:25.943830 29576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:25.943854 29563 server_base.cc:1034] running on GCE node
W20260809 03:32:25.943884 29571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:25.944285 29563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:25.944566 29563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:25.945714 29563 hybrid_clock.cc:648] HybridClock initialized: now 1786246345945692 us; error 39 us; skew 500 ppm
Aug 09 03:32:25 dist-test-slave-gkw7 krb5kdc[29383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246345, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:25.948261 29563 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:32:25.948727 29563 webserver.cc:460] Webserver started at http://127.27.40.129:39863/ using document root <none> and password file <none>
I20260809 03:32:25.948977 29563 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:25.949021 29563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:25.949198 29563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:25.950279 29563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "30a6bbc7e026479481e557592152f83c"
format_stamp: "Formatted at 2026-08-09 03:32:25 on dist-test-slave-gkw7"
server_key: "e9be1362e6b1536f3f7daeed4d383de9"
server_key_iv: "b8ce9f5a7e2fc446d4989e03d17ebc42"
server_key_version: "encryptionkey@0"
I20260809 03:32:25.950621 29563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "30a6bbc7e026479481e557592152f83c"
format_stamp: "Formatted at 2026-08-09 03:32:25 on dist-test-slave-gkw7"
server_key: "e9be1362e6b1536f3f7daeed4d383de9"
server_key_iv: "b8ce9f5a7e2fc446d4989e03d17ebc42"
server_key_version: "encryptionkey@0"
I20260809 03:32:25.952011 29563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 03:32:25.952750 29588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:25.952916 29563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:32:25.953022 29563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "30a6bbc7e026479481e557592152f83c"
format_stamp: "Formatted at 2026-08-09 03:32:25 on dist-test-slave-gkw7"
server_key: "e9be1362e6b1536f3f7daeed4d383de9"
server_key_iv: "b8ce9f5a7e2fc446d4989e03d17ebc42"
server_key_version: "encryptionkey@0"
I20260809 03:32:25.953095 29563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:25.959883 29563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:25.961714 29563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:25.961880 29563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:25.962491 29563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:25.962543 29563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:25.962572 29563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:25.962594 29563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:32:25 dist-test-slave-gkw7 krb5kdc[29383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246345, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:25.969911 29563 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:45969
I20260809 03:32:25.970355 29563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:32:25.970745 29784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:45969 every 8 connection(s)
I20260809 03:32:25.972077 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:25.962875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:42719 (local address 127.27.40.190:41295)
0809 03:32:25.962941 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:25.962943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:25.963158 (+ 215us) server_negotiation.cc:407] Connection header received
0809 03:32:25.963687 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:25.963689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:25.963703 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:25.963768 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:25.964452 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:25.964981 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:25.966104 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:25.966322 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:25.966402 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:25.968767 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:25.968769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:25.968769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:25.968777 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:25.970569 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:25.971048 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:25.971049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:25.971049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:25.971134 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:25.971410 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:25.971411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:25.971411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:25.971534 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:25.971608 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:25.971890 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 03:32:25.971985 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 03:32:25.972234 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:25.962859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:41295 (local address 127.27.40.129:42719)
0809 03:32:25.963020 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:25.963054 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:25.963561 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:25.963836 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:25.963840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:25.964169 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:25.964350 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:25.964355 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:25.965082 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:25.965083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:25.965996 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:25.965999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:25.966414 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:25.966415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:25.966535 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:25.966998 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:25.967006 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:25.968652 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:25.970692 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:25.970697 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:25.970698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:25.970971 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:25.971201 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:25.971203 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:25.971203 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:25.971338 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:25.971613 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:25.971619 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:25.971753 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 03:32:25.971855 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 03:32:25.972009 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260809 03:32:25.972780 29788 heartbeater.cc:347] Connected to a master server at 127.27.40.190:41295
I20260809 03:32:25.972888 29788 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:25.973058 29788 heartbeater.cc:511] Master 127.27.40.190:41295 requested a full tablet report, sending...
I20260809 03:32:25.973711 29455 ts_manager.cc:194] Registered new tserver with Master: 30a6bbc7e026479481e557592152f83c (127.27.40.129:45969)
I20260809 03:32:25.974363 29455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:42719
I20260809 03:32:25.977994 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 29563
I20260809 03:32:25.978147 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:32:25.978339 27810 external_mini_cluster.cc:1428] Setting key c3943948cc9b7945155784c7671217c3
I20260809 03:32:25.979259 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:25.973906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:46991 (local address 127.27.40.190:41295)
0809 03:32:25.973946 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:25.973948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:25.974051 (+ 103us) server_negotiation.cc:407] Connection header received
0809 03:32:25.974123 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:25.974124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:25.974134 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:25.974177 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:25.974550 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:25.975041 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:25.975737 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:25.975941 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:25.976025 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:25.976846 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:25.976848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:25.976848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:25.976855 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:25.978076 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:25.978477 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:25.978478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:25.978479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:25.978546 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:25.978778 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:25.978779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:25.978780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:25.978904 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:25.978972 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:25.979136 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 03:32:25.979203 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 03:32:25.979382 29802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:25.973856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:41295 (local address 127.27.40.129:46991)
0809 03:32:25.973985 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:25.973994 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:25.974065 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:25.974199 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:25.974200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:25.974373 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:25.974429 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:25.974430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:25.975135 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:25.975136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:25.975651 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:25.975653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:25.976028 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:25.976029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:25.976108 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:25.976479 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:25.976483 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:25.976759 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:25.978166 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:25.978167 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:25.978168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:25.978394 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:25.978634 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:25.978635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:25.978635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:25.978712 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:25.978976 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:25.978976 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:25.979043 (+ 67us) client_negotiation.cc:769] Sending connection context
0809 03:32:25.979118 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 03:32:25.979232 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:32:26.004434 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:41295
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:46531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:26.086241 29808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:26.086491 29808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:26.086532 29808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:26.087689 29808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:26.087759 29808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:26.087793 29808 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:32:26.087833 29808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:32:26.089380 29808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:46531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.40.190:41295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:26.089689 29808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:26.089926 29808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:26.094959 29819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:26.094935 29823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:26.094942 29820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:26.095264 29808 server_base.cc:1034] running on GCE node
I20260809 03:32:26.095435 29808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:26.095659 29808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:26.096848 29808 hybrid_clock.cc:648] HybridClock initialized: now 1786246346096831 us; error 27 us; skew 500 ppm
Aug 09 03:32:26 dist-test-slave-gkw7 krb5kdc[29383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246346, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:26.099587 29808 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:32:26.100039 29808 webserver.cc:460] Webserver started at http://127.27.40.130:43253/ using document root <none> and password file <none>
I20260809 03:32:26.100299 29808 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:26.100374 29808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:26.100531 29808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:26.101672 29808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "63020353344542d3a52f40dd92d21b47"
format_stamp: "Formatted at 2026-08-09 03:32:26 on dist-test-slave-gkw7"
server_key: "035bba80ad467b6698601b84e7eab154"
server_key_iv: "b4e7163f5b9331c368db9e0de21197c0"
server_key_version: "encryptionkey@0"
I20260809 03:32:26.102083 29808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "63020353344542d3a52f40dd92d21b47"
format_stamp: "Formatted at 2026-08-09 03:32:26 on dist-test-slave-gkw7"
server_key: "035bba80ad467b6698601b84e7eab154"
server_key_iv: "b4e7163f5b9331c368db9e0de21197c0"
server_key_version: "encryptionkey@0"
I20260809 03:32:26.103461 29808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:32:26.104197 29837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:26.104360 29808 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:26.104470 29808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "63020353344542d3a52f40dd92d21b47"
format_stamp: "Formatted at 2026-08-09 03:32:26 on dist-test-slave-gkw7"
server_key: "035bba80ad467b6698601b84e7eab154"
server_key_iv: "b4e7163f5b9331c368db9e0de21197c0"
server_key_version: "encryptionkey@0"
I20260809 03:32:26.104544 29808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:26.119740 29808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:26.121811 29808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:26.121977 29808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:26.122498 29808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:26.122563 29808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:26.122603 29808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:26.122653 29808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:26.129482 29808 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:45363
I20260809 03:32:26.129601 30020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:45363 every 8 connection(s)
I20260809 03:32:26.129928 29808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
Aug 09 03:32:26 dist-test-slave-gkw7 krb5kdc[29383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246346, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:26.133963 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.123121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:51539 (local address 127.27.40.190:41295)
0809 03:32:26.123196 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:26.123199 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:26.123399 (+ 200us) server_negotiation.cc:407] Connection header received
0809 03:32:26.124029 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:26.124039 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:26.124053 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:26.124120 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:26.125169 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.126076 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.127812 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.128165 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.128244 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.131148 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:26.131149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:26.131150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:26.131157 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:26.132675 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:26.133064 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:26.133065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:26.133065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:26.133119 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:26.133308 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:26.133308 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:26.133308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:26.133414 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:26.133465 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:26.133832 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 03:32:26.133900 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 03:32:26.134254 29855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.123054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:41295 (local address 127.27.40.130:51539)
0809 03:32:26.123252 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:26.123307 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:26.123923 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:26.124260 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:26.124265 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:26.124681 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:26.124997 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.125004 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.126210 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.126212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.127707 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.127710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.128283 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.128284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.128512 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.129070 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:26.129079 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:26.131049 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:26.132752 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:26.132758 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:26.132760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:26.132989 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:26.133157 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:26.133157 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:26.133158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:26.133256 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:26.133487 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:26.133495 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:26.133692 (+ 197us) client_negotiation.cc:769] Sending connection context
0809 03:32:26.133803 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 03:32:26.133932 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260809 03:32:26.134835 30021 heartbeater.cc:347] Connected to a master server at 127.27.40.190:41295
I20260809 03:32:26.134929 30021 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:26.135149 30021 heartbeater.cc:511] Master 127.27.40.190:41295 requested a full tablet report, sending...
I20260809 03:32:26.135541 29454 ts_manager.cc:194] Registered new tserver with Master: 63020353344542d3a52f40dd92d21b47 (127.27.40.130:45363)
I20260809 03:32:26.135979 29454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:51539
I20260809 03:32:26.139880 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 29808
I20260809 03:32:26.140000 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:32:26.140213 27810 external_mini_cluster.cc:1428] Setting key 297190aa876c514cb24a31aecdc09b7e
I20260809 03:32:26.142552 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.136164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:37627 (local address 127.27.40.190:41295)
0809 03:32:26.136325 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:26.136327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:26.136334 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:32:26.136397 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:26.136398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:26.136406 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:26.136452 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:26.136818 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.137284 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.138144 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.138368 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.138439 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.139396 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:26.139398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:26.139398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:26.139404 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:26.141263 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:26.141746 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:26.141747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:26.141747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:26.141811 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:26.142029 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:26.142030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:26.142030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:26.142197 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:26.142250 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:26.142419 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 03:32:26.142487 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260809 03:32:26.142665 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.136104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:41295 (local address 127.27.40.130:37627)
0809 03:32:26.136236 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:26.136247 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:26.136326 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:26.136458 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:26.136459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:26.136674 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:26.136738 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.136740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.137378 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.137379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.137996 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.137998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.138457 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.138458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.138526 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.138949 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:26.138954 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:26.139302 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:26.141374 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:26.141375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:26.141376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:26.141666 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:26.141879 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:26.141880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:26.141881 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:26.141969 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:26.142262 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:26.142263 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:26.142320 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 03:32:26.142399 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 03:32:26.142522 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:32:26.168853 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:41295
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:46531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:26.254796 30039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:26.255017 30039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:26.255051 30039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:26.256328 30039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:26.256376 30039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:26.256426 30039 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:32:26.256475 30039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:32:26.257978 30039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:46531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.40.190:41295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:26.258260 30039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:26.258481 30039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:26.263738 30047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:26.263784 30049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:26.263775 30046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:26.264164 30039 server_base.cc:1034] running on GCE node
I20260809 03:32:26.264345 30039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:26.264542 30039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:26.265695 30039 hybrid_clock.cc:648] HybridClock initialized: now 1786246346265663 us; error 34 us; skew 500 ppm
Aug 09 03:32:26 dist-test-slave-gkw7 krb5kdc[29383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246346, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:26.268163 30039 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:32:26.268611 30039 webserver.cc:460] Webserver started at http://127.27.40.131:42089/ using document root <none> and password file <none>
I20260809 03:32:26.268872 30039 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:26.268919 30039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:26.269083 30039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:26.270254 30039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "c1c042cf3d2e48d68dcf673b783bcc95"
format_stamp: "Formatted at 2026-08-09 03:32:26 on dist-test-slave-gkw7"
server_key: "86e5f421c016e3bd6aeb0d014b42f4c8"
server_key_iv: "e3386205ca1c73b83b6cf5b379f4ce20"
server_key_version: "encryptionkey@0"
I20260809 03:32:26.270630 30039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "c1c042cf3d2e48d68dcf673b783bcc95"
format_stamp: "Formatted at 2026-08-09 03:32:26 on dist-test-slave-gkw7"
server_key: "86e5f421c016e3bd6aeb0d014b42f4c8"
server_key_iv: "e3386205ca1c73b83b6cf5b379f4ce20"
server_key_version: "encryptionkey@0"
I20260809 03:32:26.272046 30039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:32:26.272953 30060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:26.273190 30039 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:26.273315 30039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "c1c042cf3d2e48d68dcf673b783bcc95"
format_stamp: "Formatted at 2026-08-09 03:32:26 on dist-test-slave-gkw7"
server_key: "86e5f421c016e3bd6aeb0d014b42f4c8"
server_key_iv: "e3386205ca1c73b83b6cf5b379f4ce20"
server_key_version: "encryptionkey@0"
I20260809 03:32:26.273396 30039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:26.303758 30039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:26.306329 30039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:26.306490 30039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:26.307052 30039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:26.307090 30039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:26.307116 30039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:26.307132 30039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:32:26 dist-test-slave-gkw7 krb5kdc[29383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246346, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:26.316543 30039 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:39719
I20260809 03:32:26.316617 30248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:39719 every 8 connection(s)
I20260809 03:32:26.316934 30039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:32:26.318800 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.307657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:57781 (local address 127.27.40.190:41295)
0809 03:32:26.307738 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:26.307741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:26.308000 (+ 259us) server_negotiation.cc:407] Connection header received
0809 03:32:26.308821 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:26.308823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:26.308838 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:26.308919 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:26.310056 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.310985 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.312237 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.312591 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.312684 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.315740 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:26.315742 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:26.315742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:26.315750 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:26.317321 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:26.317802 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:26.317803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:26.317804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:26.317869 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:26.318133 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:26.318134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:26.318134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:26.318258 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:26.318331 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:26.318617 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 03:32:26.318713 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 03:32:26.318974 30074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.307621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:41295 (local address 127.27.40.131:57781)
0809 03:32:26.307872 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:26.307918 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:26.308661 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:26.309109 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:26.309114 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:26.309612 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:26.309927 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.309934 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.311129 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.311130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.312152 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.312156 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.312709 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.312710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.312880 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.313668 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:26.313678 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:26.315616 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:26.317400 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:26.317405 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:26.317407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:26.317723 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:26.317939 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:26.317940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:26.317941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:26.318058 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:26.318335 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:26.318342 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:26.318481 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 03:32:26.318581 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 03:32:26.318744 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":113,"threads_started":1}
I20260809 03:32:26.319433 30249 heartbeater.cc:347] Connected to a master server at 127.27.40.190:41295
I20260809 03:32:26.319521 30249 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:26.319715 30249 heartbeater.cc:511] Master 127.27.40.190:41295 requested a full tablet report, sending...
I20260809 03:32:26.320078 29455 ts_manager.cc:194] Registered new tserver with Master: c1c042cf3d2e48d68dcf673b783bcc95 (127.27.40.131:39719)
I20260809 03:32:26.320633 29455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:57781
I20260809 03:32:26.325579 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 30039
I20260809 03:32:26.325672 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:32:26.325837 27810 external_mini_cluster.cc:1428] Setting key accfde0bea3cc99740c1272b6168dee2
I20260809 03:32:26.326741 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:32:26.327050 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.320803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:38219 (local address 127.27.40.190:41295)
0809 03:32:26.320847 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:26.320849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:26.321010 (+ 161us) server_negotiation.cc:407] Connection header received
0809 03:32:26.321060 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:26.321061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:26.321072 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:26.321118 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:26.321761 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.322240 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.323005 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.323212 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.323287 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.324311 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:26.324312 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:26.324312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:26.324320 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:26.325833 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:26.326211 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:26.326213 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:26.326213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:26.326278 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:26.326496 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:26.326496 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:26.326497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:26.326641 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:26.326708 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:26.326880 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 03:32:26.326954 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 03:32:26.327064 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.320659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:41295 (local address 127.27.40.131:38219)
0809 03:32:26.320921 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:26.320930 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:26.321004 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:26.321166 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:26.321167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:26.321486 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:26.321674 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.321676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.322402 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.322403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.322907 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.322908 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.323286 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.323287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.323438 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.323921 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:26.323926 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:26.324206 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:26.325911 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:26.325912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:26.325912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:26.326128 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:26.326354 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:26.326355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:26.326355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:26.326424 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:26.326713 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:26.326714 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:26.326762 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 03:32:26.326848 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 03:32:26.326958 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":161,"threads_started":1}
Aug 09 03:32:26 dist-test-slave-gkw7 krb5kdc[29383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246346, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 03:32:26 dist-test-slave-gkw7 krb5kdc[29383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246346, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:26.352006 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.344667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38154 (local address 127.27.40.190:41295)
0809 03:32:26.344780 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:26.344782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:26.344793 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:32:26.344820 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:26.344821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:26.344830 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:26.344923 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:26.345716 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.346216 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.346992 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.347188 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.347267 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.349489 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:26.349491 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:26.349491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:26.349498 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:26.351010 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:26.351351 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:26.351353 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:26.351353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:26.351413 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:26.351588 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:26.351589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:26.351589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:26.351699 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:26.351776 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:26.351869 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 03:32:26.351933 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 03:32:26.353926 29455 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38154:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:32:26.355711 29455 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:26.362946 30275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.358231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:45969 (local address 127.0.0.1:59068)
0809 03:32:26.358644 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:26.358655 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:26.358715 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:26.359433 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:26.359434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:26.359436 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:26.359475 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.359476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.360804 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.360805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.361836 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.361838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.362645 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.362646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.362704 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.362726 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 03:32:26.362764 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 03:32:26.362824 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":28,"thread_start_us":83,"threads_started":1}
I20260809 03:32:26.362947 30271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.358415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:45363 (local address 127.0.0.1:33858)
0809 03:32:26.358768 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:26.358796 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:26.358836 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:26.359123 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:26.359127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:26.359129 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:26.359209 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.359211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.360270 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.360271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.361372 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.361375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.362604 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.362605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.362696 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.362713 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 03:32:26.362791 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 03:32:26.362828 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":3,"thread_start_us":65,"threads_started":1}
I20260809 03:32:26.363337 30276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.358405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59068 (local address 127.27.40.129:45969)
0809 03:32:26.359272 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:26.359277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:26.359288 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:32:26.359326 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:26.359328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:26.359345 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:26.359430 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:26.359577 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.360705 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.361934 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.362527 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.363102 (+ 575us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.363172 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:26.363199 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 03:32:26.363242 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":828,"thread_start_us":85,"threads_started":1}
I20260809 03:32:26.363534 30273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.358217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:39719 (local address 127.0.0.1:42452)
0809 03:32:26.358457 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:26.358500 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:26.358594 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:26.359507 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:26.359508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:26.359511 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:26.359587 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.359588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.360818 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.360819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.362087 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.362091 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.363301 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.363302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.363370 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.363374 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:26.363419 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 03:32:26.363456 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":3,"thread_start_us":50,"threads_started":1}
I20260809 03:32:26.363831 30274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.358326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42452 (local address 127.27.40.131:39719)
0809 03:32:26.358952 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:26.358956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:26.358965 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:26.359002 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:26.359005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:26.359020 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:26.359089 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:26.359654 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.360729 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.362328 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.363126 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.363207 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.363266 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:26.363665 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 03:32:26.363714 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":113,"threads_started":1}
I20260809 03:32:26.364035 30277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.358682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33858 (local address 127.27.40.130:45363)
0809 03:32:26.358875 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:26.358880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:26.358892 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:32:26.358958 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:26.358960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:26.358980 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:26.359068 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:26.359303 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.360173 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.361465 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.362508 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.362588 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.362648 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:26.362817 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 03:32:26.362857 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":96,"threads_started":1}
I20260809 03:32:26.364100 29671 tablet_service.cc:1467] Processing CreateTablet for tablet 960afbde0f5749a7814fd685569560b0 (DEFAULT_TABLE table=test-table [id=5e29bb6cf28b48d19cb631651b894cc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:26.364513 29671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 960afbde0f5749a7814fd685569560b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:26.364498 30120 tablet_service.cc:1467] Processing CreateTablet for tablet 960afbde0f5749a7814fd685569560b0 (DEFAULT_TABLE table=test-table [id=5e29bb6cf28b48d19cb631651b894cc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:26.364822 30120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 960afbde0f5749a7814fd685569560b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:26.367736 30280 tablet_bootstrap.cc:492] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95: Bootstrap starting.
I20260809 03:32:26.367933 30281 tablet_bootstrap.cc:492] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c: Bootstrap starting.
I20260809 03:32:26.368369 29914 tablet_service.cc:1467] Processing CreateTablet for tablet 960afbde0f5749a7814fd685569560b0 (DEFAULT_TABLE table=test-table [id=5e29bb6cf28b48d19cb631651b894cc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:26.368703 29914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 960afbde0f5749a7814fd685569560b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:26.368763 30280 tablet_bootstrap.cc:654] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:26.368793 30281 tablet_bootstrap.cc:654] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:26.369127 30281 log.cc:824] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:26.369511 30280 log.cc:824] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:26.370028 30281 tablet_bootstrap.cc:492] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c: No bootstrap required, opened a new log
I20260809 03:32:26.370177 30281 ts_tablet_manager.cc:1397] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:32:26.370611 30280 tablet_bootstrap.cc:492] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95: No bootstrap required, opened a new log
I20260809 03:32:26.370692 30280 ts_tablet_manager.cc:1397] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 03:32:26.371742 30281 raft_consensus.cc:348] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } }
I20260809 03:32:26.371898 30281 raft_consensus.cc:374] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:26.371943 30281 raft_consensus.cc:729] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a6bbc7e026479481e557592152f83c, State: Initialized, Role: FOLLOWER
I20260809 03:32:26.372092 30281 consensus_queue.cc:260] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } }
I20260809 03:32:26.372382 30281 ts_tablet_manager.cc:1428] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:32:26.372520 29788 heartbeater.cc:503] Master 127.27.40.190:41295 was elected leader, sending a full tablet report...
I20260809 03:32:26.373195 30288 tablet_bootstrap.cc:492] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47: Bootstrap starting.
I20260809 03:32:26.373263 30280 raft_consensus.cc:348] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } }
I20260809 03:32:26.373422 30280 raft_consensus.cc:374] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:26.373461 30280 raft_consensus.cc:729] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1c042cf3d2e48d68dcf673b783bcc95, State: Initialized, Role: FOLLOWER
I20260809 03:32:26.373602 30280 consensus_queue.cc:260] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } }
I20260809 03:32:26.373931 30280 ts_tablet_manager.cc:1428] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260809 03:32:26.373946 30249 heartbeater.cc:503] Master 127.27.40.190:41295 was elected leader, sending a full tablet report...
I20260809 03:32:26.374022 30288 tablet_bootstrap.cc:654] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:26.374364 30288 log.cc:824] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:26.375208 30288 tablet_bootstrap.cc:492] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47: No bootstrap required, opened a new log
I20260809 03:32:26.375263 30288 ts_tablet_manager.cc:1397] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 03:32:26.377017 30288 raft_consensus.cc:348] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } }
I20260809 03:32:26.377136 30288 raft_consensus.cc:374] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:26.377158 30288 raft_consensus.cc:729] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63020353344542d3a52f40dd92d21b47, State: Initialized, Role: FOLLOWER
I20260809 03:32:26.377296 30288 consensus_queue.cc:260] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } }
I20260809 03:32:26.377660 30288 ts_tablet_manager.cc:1428] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 03:32:26.377672 30021 heartbeater.cc:503] Master 127.27.40.190:41295 was elected leader, sending a full tablet report...
W20260809 03:32:26.380473 30023 tablet.cc:2369] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:32:26.442852 30294 raft_consensus.cc:482] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:26.443027 30294 raft_consensus.cc:504] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } }
I20260809 03:32:26.443459 30294 leader_election.cc:288] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30a6bbc7e026479481e557592152f83c (127.27.40.129:45969), c1c042cf3d2e48d68dcf673b783bcc95 (127.27.40.131:39719)
I20260809 03:32:26.446650 30274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.443718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:35551 (local address 127.27.40.131:39719)
0809 03:32:26.443774 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:26.443776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:26.443807 (+ 31us) server_negotiation.cc:407] Connection header received
0809 03:32:26.443837 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:26.443838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:26.443848 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:26.443894 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:26.444146 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.444801 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.445921 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.446295 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.446382 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.446392 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:26.446552 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 03:32:26.446593 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 03:32:26.446646 29855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.443638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:39719 (local address 127.27.40.130:35551)
0809 03:32:26.443699 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:26.443711 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:26.443758 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:26.443946 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:26.443947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:26.443948 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:26.444043 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.444045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.444874 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.444875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.445834 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.445836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.446380 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.446381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.446468 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.446471 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:32:26.446532 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 03:32:26.446573 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":4}
I20260809 03:32:26.446882 30295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.443692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:45969 (local address 127.27.40.130:41843)
0809 03:32:26.443936 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:26.443946 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:26.444013 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:26.444143 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:26.444144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:26.444145 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:26.444397 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.444400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.445225 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.445226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.445991 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.445993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.446573 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.446574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.446648 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.446651 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:32:26.446715 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 03:32:26.446776 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":189,"threads_started":1}
I20260809 03:32:26.446976 30276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.443804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:41843 (local address 127.27.40.129:45969)
0809 03:32:26.443855 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:26.443858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:26.444027 (+ 169us) server_negotiation.cc:407] Connection header received
0809 03:32:26.444060 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:26.444061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:26.444071 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:26.444127 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:26.444468 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.445042 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.446085 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.446481 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.446572 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.446582 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:26.446846 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 03:32:26.446892 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 03:32:26.446961 30160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "960afbde0f5749a7814fd685569560b0" candidate_uuid: "63020353344542d3a52f40dd92d21b47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" is_pre_election: true
I20260809 03:32:26.447144 30160 raft_consensus.cc:2454] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63020353344542d3a52f40dd92d21b47 in term 0.
I20260809 03:32:26.447253 29698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "960afbde0f5749a7814fd685569560b0" candidate_uuid: "63020353344542d3a52f40dd92d21b47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30a6bbc7e026479481e557592152f83c" is_pre_election: true
I20260809 03:32:26.447422 29698 raft_consensus.cc:2454] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63020353344542d3a52f40dd92d21b47 in term 0.
I20260809 03:32:26.447432 29842 leader_election.cc:302] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63020353344542d3a52f40dd92d21b47, c1c042cf3d2e48d68dcf673b783bcc95; no voters:
I20260809 03:32:26.447594 30294 raft_consensus.cc:2781] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:26.447655 30294 raft_consensus.cc:482] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:26.447697 30294 raft_consensus.cc:3037] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:26.448902 30294 raft_consensus.cc:504] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } }
I20260809 03:32:26.449046 30294 leader_election.cc:288] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [CANDIDATE]: Term 1 election: Requested vote from peers 30a6bbc7e026479481e557592152f83c (127.27.40.129:45969), c1c042cf3d2e48d68dcf673b783bcc95 (127.27.40.131:39719)
I20260809 03:32:26.449218 29698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "960afbde0f5749a7814fd685569560b0" candidate_uuid: "63020353344542d3a52f40dd92d21b47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30a6bbc7e026479481e557592152f83c"
I20260809 03:32:26.449223 30160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "960afbde0f5749a7814fd685569560b0" candidate_uuid: "63020353344542d3a52f40dd92d21b47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1c042cf3d2e48d68dcf673b783bcc95"
I20260809 03:32:26.449297 30160 raft_consensus.cc:3037] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:26.449297 29698 raft_consensus.cc:3037] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:26.450335 29698 raft_consensus.cc:2454] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63020353344542d3a52f40dd92d21b47 in term 1.
I20260809 03:32:26.450331 30160 raft_consensus.cc:2454] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63020353344542d3a52f40dd92d21b47 in term 1.
I20260809 03:32:26.450519 29846 leader_election.cc:302] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30a6bbc7e026479481e557592152f83c, 63020353344542d3a52f40dd92d21b47; no voters:
I20260809 03:32:26.450626 30294 raft_consensus.cc:2781] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:26.450831 30294 raft_consensus.cc:686] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [term 1 LEADER]: Becoming Leader. State: Replica: 63020353344542d3a52f40dd92d21b47, State: Running, Role: LEADER
I20260809 03:32:26.450949 30294 consensus_queue.cc:237] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } }
I20260809 03:32:26.451746 29455 catalog_manager.cc:5495] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 63020353344542d3a52f40dd92d21b47 (127.27.40.130). New cstate: current_term: 1 leader_uuid: "63020353344542d3a52f40dd92d21b47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } health_report { overall_health: HEALTHY } } }
W20260809 03:32:26.471570 29789 tablet.cc:2369] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:32:26.479316 30277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.476875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33872 (local address 127.27.40.130:45363)
0809 03:32:26.476932 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:26.476934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:26.476946 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:32:26.477042 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:26.477043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:26.477060 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:26.477132 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:26.477300 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.477868 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.478620 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.478809 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.478860 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.479027 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:26.479118 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:26.479174 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:26.479222 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 03:32:26.479242 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 03:32:26.481213 29698 raft_consensus.cc:1261] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c [term 1 FOLLOWER]: Refusing update from remote peer 63020353344542d3a52f40dd92d21b47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:26.481215 30160 raft_consensus.cc:1261] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95 [term 1 FOLLOWER]: Refusing update from remote peer 63020353344542d3a52f40dd92d21b47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:26.481563 30294 consensus_queue.cc:1035] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:26.481729 30296 consensus_queue.cc:1035] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:26.484936 30300 mvcc.cc:205] Tried to move back new op lower bound from 7316465035184050176 to 7316465035063635968. Current Snapshot: MvccSnapshot[applied={T|T < 7316465035184050176}]
I20260809 03:32:26.485271 30301 mvcc.cc:205] Tried to move back new op lower bound from 7316465035184050176 to 7316465035063635968. Current Snapshot: MvccSnapshot[applied={T|T < 7316465035184050176}]
I20260809 03:32:26.488018 29454 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:26.488125 29454 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:26.488996 29454 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae: Sending DeleteTablet for 3 replicas of tablet 960afbde0f5749a7814fd685569560b0
I20260809 03:32:26.489354 29671 tablet_service.cc:1514] Processing DeleteTablet for tablet 960afbde0f5749a7814fd685569560b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:26 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:59068
I20260809 03:32:26.489428 30120 tablet_service.cc:1514] Processing DeleteTablet for tablet 960afbde0f5749a7814fd685569560b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:26 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:42452
I20260809 03:32:26.489703 29914 tablet_service.cc:1514] Processing DeleteTablet for tablet 960afbde0f5749a7814fd685569560b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:26 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:33858
I20260809 03:32:26.489856 30308 tablet_replica.cc:330] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c: stopping tablet replica
I20260809 03:32:26.489959 30308 raft_consensus.cc:2229] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:26.490005 30310 tablet_replica.cc:330] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47: stopping tablet replica
I20260809 03:32:26.489972 29454 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38154:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 03:32:26.490074 30308 raft_consensus.cc:2258] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:26.490082 30310 raft_consensus.cc:2229] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [term 1 LEADER]: Raft consensus shutting down.
W20260809 03:32:26.490141 29454 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:26.490167 30310 raft_consensus.cc:2258] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:26.490355 30309 tablet_replica.cc:330] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95: stopping tablet replica
I20260809 03:32:26.490446 30309 raft_consensus.cc:2229] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:26.490548 30309 raft_consensus.cc:2258] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:26.490597 30308 ts_tablet_manager.cc:1905] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:26.490955 30310 ts_tablet_manager.cc:1905] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:26.491959 29671 tablet_service.cc:1467] Processing CreateTablet for tablet 09e88905ca304c6988ee69260e035873 (DEFAULT_TABLE table=test-table [id=4487a5a1bc3546469a422ade0d63d298]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:26.492081 29671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09e88905ca304c6988ee69260e035873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:26.492677 30308 ts_tablet_manager.cc:1918] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:26.492743 30308 log.cc:1196] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/wal/wals/960afbde0f5749a7814fd685569560b0
I20260809 03:32:26.493001 30308 ts_tablet_manager.cc:1939] T 960afbde0f5749a7814fd685569560b0 P 30a6bbc7e026479481e557592152f83c: Deleting consensus metadata
I20260809 03:32:26.493105 30310 ts_tablet_manager.cc:1918] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:26.493166 30310 log.cc:1196] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/960afbde0f5749a7814fd685569560b0
I20260809 03:32:26.493414 30310 ts_tablet_manager.cc:1939] T 960afbde0f5749a7814fd685569560b0 P 63020353344542d3a52f40dd92d21b47: Deleting consensus metadata
I20260809 03:32:26.493468 29436 catalog_manager.cc:4841] TS 30a6bbc7e026479481e557592152f83c (127.27.40.129:45969): tablet 960afbde0f5749a7814fd685569560b0 (table test-table [id=5e29bb6cf28b48d19cb631651b894cc5]) successfully deleted
I20260809 03:32:26.493500 30281 tablet_bootstrap.cc:492] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c: Bootstrap starting.
I20260809 03:32:26.494138 29914 tablet_service.cc:1467] Processing CreateTablet for tablet 09e88905ca304c6988ee69260e035873 (DEFAULT_TABLE table=test-table [id=4487a5a1bc3546469a422ade0d63d298]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:26.494237 30281 tablet_bootstrap.cc:654] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:26.494271 29914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09e88905ca304c6988ee69260e035873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:26.494534 30120 tablet_service.cc:1467] Processing CreateTablet for tablet 09e88905ca304c6988ee69260e035873 (DEFAULT_TABLE table=test-table [id=4487a5a1bc3546469a422ade0d63d298]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:26.494652 30120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09e88905ca304c6988ee69260e035873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:26.495143 29435 catalog_manager.cc:4841] TS 63020353344542d3a52f40dd92d21b47 (127.27.40.130:45363): tablet 960afbde0f5749a7814fd685569560b0 (table test-table [id=5e29bb6cf28b48d19cb631651b894cc5]) successfully deleted
I20260809 03:32:26.495735 30288 tablet_bootstrap.cc:492] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47: Bootstrap starting.
I20260809 03:32:26.496353 30280 tablet_bootstrap.cc:492] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95: Bootstrap starting.
I20260809 03:32:26.496402 30288 tablet_bootstrap.cc:654] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:26.497044 30280 tablet_bootstrap.cc:654] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:26.497645 30280 tablet_bootstrap.cc:492] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95: No bootstrap required, opened a new log
I20260809 03:32:26.497690 30280 ts_tablet_manager.cc:1397] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:26.497819 30288 tablet_bootstrap.cc:492] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47: No bootstrap required, opened a new log
I20260809 03:32:26.497877 30288 ts_tablet_manager.cc:1397] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:32:26.497860 30280 raft_consensus.cc:348] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:26.497938 30280 raft_consensus.cc:374] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:26.497967 30280 raft_consensus.cc:729] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1c042cf3d2e48d68dcf673b783bcc95, State: Initialized, Role: FOLLOWER
I20260809 03:32:26.498018 30280 consensus_queue.cc:260] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:26.498104 30280 ts_tablet_manager.cc:1428] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:26.498260 30309 ts_tablet_manager.cc:1905] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:26.498207 30288 raft_consensus.cc:348] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:26.498339 30288 raft_consensus.cc:374] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:26.498369 30288 raft_consensus.cc:729] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63020353344542d3a52f40dd92d21b47, State: Initialized, Role: FOLLOWER
I20260809 03:32:26.498410 30288 consensus_queue.cc:260] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:26.498551 30288 ts_tablet_manager.cc:1428] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:26.499971 30309 ts_tablet_manager.cc:1918] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:26.500044 30309 log.cc:1196] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/960afbde0f5749a7814fd685569560b0
I20260809 03:32:26.500102 30281 tablet_bootstrap.cc:492] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c: No bootstrap required, opened a new log
I20260809 03:32:26.500146 30281 ts_tablet_manager.cc:1397] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260809 03:32:26.500317 30309 ts_tablet_manager.cc:1939] T 960afbde0f5749a7814fd685569560b0 P c1c042cf3d2e48d68dcf673b783bcc95: Deleting consensus metadata
I20260809 03:32:26.500401 30281 raft_consensus.cc:348] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:26.500449 30281 raft_consensus.cc:374] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:26.500463 30281 raft_consensus.cc:729] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a6bbc7e026479481e557592152f83c, State: Initialized, Role: FOLLOWER
I20260809 03:32:26.500523 30281 consensus_queue.cc:260] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:26.500659 30281 ts_tablet_manager.cc:1428] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:26.500818 29435 catalog_manager.cc:4841] TS c1c042cf3d2e48d68dcf673b783bcc95 (127.27.40.131:39719): tablet 960afbde0f5749a7814fd685569560b0 (table test-table [id=5e29bb6cf28b48d19cb631651b894cc5]) successfully deleted
W20260809 03:32:26.567806 30250 tablet.cc:2369] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:32:26.855604 30285 raft_consensus.cc:482] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:26.855715 30285 raft_consensus.cc:504] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:26.856071 30285 leader_election.cc:288] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63020353344542d3a52f40dd92d21b47 (127.27.40.130:45363), c1c042cf3d2e48d68dcf673b783bcc95 (127.27.40.131:39719)
I20260809 03:32:26.859529 30274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.856585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:51875 (local address 127.27.40.131:39719)
0809 03:32:26.856679 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:26.856681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:26.856732 (+ 51us) server_negotiation.cc:407] Connection header received
0809 03:32:26.856801 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:26.856802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:26.856811 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:26.856869 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:26.857050 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.857600 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.858763 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.859138 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.859192 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.859199 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:26.859399 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 03:32:26.859446 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 03:32:26.859550 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.856504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:39719 (local address 127.27.40.129:51875)
0809 03:32:26.856676 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:26.856687 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:26.856745 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:26.856891 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:26.856891 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:26.856893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:26.856958 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.856959 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.857720 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.857721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.858669 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.858671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.859253 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.859254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.859327 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.859332 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:32:26.859389 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 03:32:26.859420 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":80,"threads_started":1}
I20260809 03:32:26.859679 30277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.856353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:33703 (local address 127.27.40.130:45363)
0809 03:32:26.856460 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:26.856462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:26.856531 (+ 69us) server_negotiation.cc:407] Connection header received
0809 03:32:26.856599 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:26.856600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:26.856615 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:26.856704 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:26.856891 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.857444 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.858832 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.859169 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.859267 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.859275 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:26.859525 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 03:32:26.859592 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 03:32:26.860080 30319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.856283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:45363 (local address 127.27.40.129:33703)
0809 03:32:26.856420 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:26.856432 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:26.856531 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:26.856718 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:26.856719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:26.856722 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:26.856809 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.856811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.857522 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.857523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.858736 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.858740 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.859295 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.859296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.859441 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.859445 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:26.859504 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 03:32:26.859542 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":38,"thread_start_us":88,"threads_started":1}
I20260809 03:32:26.860065 30160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09e88905ca304c6988ee69260e035873" candidate_uuid: "30a6bbc7e026479481e557592152f83c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" is_pre_election: true
I20260809 03:32:26.860167 30160 raft_consensus.cc:2454] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30a6bbc7e026479481e557592152f83c in term 0.
I20260809 03:32:26.860368 29592 leader_election.cc:302] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30a6bbc7e026479481e557592152f83c, c1c042cf3d2e48d68dcf673b783bcc95; no voters:
I20260809 03:32:26.860342 29947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09e88905ca304c6988ee69260e035873" candidate_uuid: "30a6bbc7e026479481e557592152f83c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63020353344542d3a52f40dd92d21b47" is_pre_election: true
I20260809 03:32:26.860538 29947 raft_consensus.cc:2454] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30a6bbc7e026479481e557592152f83c in term 0.
I20260809 03:32:26.860543 30285 raft_consensus.cc:2781] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:26.860580 30285 raft_consensus.cc:482] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:26.860601 30285 raft_consensus.cc:3037] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:26.861341 30285 raft_consensus.cc:504] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:26.861485 30285 leader_election.cc:288] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [CANDIDATE]: Term 1 election: Requested vote from peers 63020353344542d3a52f40dd92d21b47 (127.27.40.130:45363), c1c042cf3d2e48d68dcf673b783bcc95 (127.27.40.131:39719)
I20260809 03:32:26.861668 29947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09e88905ca304c6988ee69260e035873" candidate_uuid: "30a6bbc7e026479481e557592152f83c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63020353344542d3a52f40dd92d21b47"
I20260809 03:32:26.861757 29947 raft_consensus.cc:3037] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:26.861722 30160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09e88905ca304c6988ee69260e035873" candidate_uuid: "30a6bbc7e026479481e557592152f83c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1c042cf3d2e48d68dcf673b783bcc95"
I20260809 03:32:26.861840 30160 raft_consensus.cc:3037] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:26.862445 29947 raft_consensus.cc:2454] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30a6bbc7e026479481e557592152f83c in term 1.
I20260809 03:32:26.862612 30160 raft_consensus.cc:2454] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30a6bbc7e026479481e557592152f83c in term 1.
I20260809 03:32:26.862637 29592 leader_election.cc:302] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30a6bbc7e026479481e557592152f83c, 63020353344542d3a52f40dd92d21b47; no voters:
I20260809 03:32:26.862737 30285 raft_consensus.cc:2781] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:26.862879 30285 raft_consensus.cc:686] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [term 1 LEADER]: Becoming Leader. State: Replica: 30a6bbc7e026479481e557592152f83c, State: Running, Role: LEADER
I20260809 03:32:26.862933 30285 consensus_queue.cc:237] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:26.863659 29454 catalog_manager.cc:5495] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c reported cstate change: term changed from 0 to 1, leader changed from <none> to 30a6bbc7e026479481e557592152f83c (127.27.40.129). New cstate: current_term: 1 leader_uuid: "30a6bbc7e026479481e557592152f83c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } health_report { overall_health: UNKNOWN } } }
Aug 09 03:32:26 dist-test-slave-gkw7 krb5kdc[29383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246345, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:26.962380 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.954398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38170 (local address 127.27.40.190:41295)
0809 03:32:26.954622 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:26.954625 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:26.954847 (+ 222us) server_negotiation.cc:407] Connection header received
0809 03:32:26.954956 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:26.954956 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:26.954972 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:26.955102 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:26.955768 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.956340 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.957235 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.957464 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.957587 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.959850 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:26.959856 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:26.959857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:26.959863 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:26.961096 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:26.961477 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:26.961478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:26.961479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:26.961584 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:26.961798 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:26.961800 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:26.961800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:26.961917 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:26.961992 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:26.962161 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 03:32:26.962257 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":69,"threads_started":1}
I20260809 03:32:26.962471 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:26.954326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:41295 (local address 127.0.0.1:38170)
0809 03:32:26.954743 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:26.954755 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:26.954873 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:26.955123 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:26.955124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:26.955535 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:26.955654 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.955665 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.956450 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.956452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.957120 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:26.957123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:26.957584 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:26.957585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:26.957643 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:26.958209 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:26.958214 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:26.959760 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:26.961181 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:26.961201 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:26.961204 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:26.961401 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:26.961656 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:26.961657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:26.961658 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:26.961724 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:26.961994 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:26.962003 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:26.962049 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 03:32:26.962138 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 03:32:26.962325 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":77,"thread_start_us":125,"threads_started":1}
I20260809 03:32:26.964638 29454 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:38170:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 03:32:26.964916 29454 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:26.966549 29671 tablet_service.cc:1467] Processing CreateTablet for tablet 987c8d7fb8c1450f96823941ec39ad89 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d04cac5c991040aaafd68c0b127d281c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:32:26.966549 30120 tablet_service.cc:1467] Processing CreateTablet for tablet 987c8d7fb8c1450f96823941ec39ad89 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d04cac5c991040aaafd68c0b127d281c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:32:26.966548 29914 tablet_service.cc:1467] Processing CreateTablet for tablet 987c8d7fb8c1450f96823941ec39ad89 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d04cac5c991040aaafd68c0b127d281c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:32:26.966722 29671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987c8d7fb8c1450f96823941ec39ad89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:26.966732 30120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987c8d7fb8c1450f96823941ec39ad89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:26.966728 29914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987c8d7fb8c1450f96823941ec39ad89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:26.968258 30281 tablet_bootstrap.cc:492] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c: Bootstrap starting.
I20260809 03:32:26.968477 30288 tablet_bootstrap.cc:492] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47: Bootstrap starting.
I20260809 03:32:26.968842 30280 tablet_bootstrap.cc:492] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95: Bootstrap starting.
I20260809 03:32:26.968906 30281 tablet_bootstrap.cc:654] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:26.969072 30288 tablet_bootstrap.cc:654] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:26.969640 30280 tablet_bootstrap.cc:654] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:26.969882 30281 tablet_bootstrap.cc:492] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c: No bootstrap required, opened a new log
I20260809 03:32:26.969935 30281 ts_tablet_manager.cc:1397] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:32:26.970212 30288 tablet_bootstrap.cc:492] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47: No bootstrap required, opened a new log
I20260809 03:32:26.970264 30288 ts_tablet_manager.cc:1397] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:32:26.970230 30281 raft_consensus.cc:348] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:26.970307 30281 raft_consensus.cc:374] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:26.970328 30281 raft_consensus.cc:729] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a6bbc7e026479481e557592152f83c, State: Initialized, Role: FOLLOWER
I20260809 03:32:26.970388 30281 consensus_queue.cc:260] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:26.970373 30288 raft_consensus.cc:348] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:26.970432 30288 raft_consensus.cc:374] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:26.970453 30288 raft_consensus.cc:729] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63020353344542d3a52f40dd92d21b47, State: Initialized, Role: FOLLOWER
I20260809 03:32:26.970484 30281 ts_tablet_manager.cc:1428] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:26.970492 30288 consensus_queue.cc:260] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:26.970582 30288 ts_tablet_manager.cc:1428] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:26.970590 30285 tablet_replica.cc:439] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } }
I20260809 03:32:26.970657 30285 tablet_replica.cc:442] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:26.970767 30294 tablet_replica.cc:439] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } }
I20260809 03:32:26.970851 30294 tablet_replica.cc:442] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:26.970902 30280 tablet_bootstrap.cc:492] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95: No bootstrap required, opened a new log
I20260809 03:32:26.970942 30280 ts_tablet_manager.cc:1397] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 03:32:26.971225 30280 raft_consensus.cc:348] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:26.971309 30280 raft_consensus.cc:374] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:26.971345 30280 raft_consensus.cc:729] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1c042cf3d2e48d68dcf673b783bcc95, State: Initialized, Role: FOLLOWER
I20260809 03:32:26.971398 30280 consensus_queue.cc:260] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:26.971493 30280 ts_tablet_manager.cc:1428] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 03:32:26.971592 30289 tablet_replica.cc:439] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } }
I20260809 03:32:26.971777 30289 tablet_replica.cc:442] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:27.325421 30289 raft_consensus.cc:482] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:27.325505 30289 raft_consensus.cc:504] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:27.325981 30289 leader_election.cc:288] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30a6bbc7e026479481e557592152f83c (127.27.40.129:45969), 63020353344542d3a52f40dd92d21b47 (127.27.40.130:45363)
I20260809 03:32:27.329089 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:27.326203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:45363 (local address 127.27.40.131:35051)
0809 03:32:27.326405 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:27.326416 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:27.326480 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:27.326672 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:27.326673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:27.326674 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:27.326756 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:27.326757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.327392 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.327393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:27.328256 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:27.328259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.328819 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.328821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:27.328895 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:27.328899 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:27.328951 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 03:32:27.329002 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":57,"thread_start_us":60,"threads_started":1}
I20260809 03:32:27.329111 30277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:27.326290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:35051 (local address 127.27.40.130:45363)
0809 03:32:27.326352 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:27.326354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:27.326495 (+ 141us) server_negotiation.cc:407] Connection header received
0809 03:32:27.326563 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:27.326564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:27.326575 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:27.326638 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:27.326832 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.327307 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.328376 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.328736 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.328794 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:27.328802 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:27.328979 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 03:32:27.329029 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 03:32:27.329216 30345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:27.326128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:45969 (local address 127.27.40.131:42117)
0809 03:32:27.326262 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:27.326274 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:27.326394 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:27.326566 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:27.326567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:27.326569 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:27.326663 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:27.326666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.327443 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.327452 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:27.328253 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:27.328255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.328864 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.328865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:27.328929 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:27.328933 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:27.329007 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 03:32:27.329046 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260809 03:32:27.329387 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:27.326304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:42117 (local address 127.27.40.129:45969)
0809 03:32:27.326409 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:27.326411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:27.326442 (+ 31us) server_negotiation.cc:407] Connection header received
0809 03:32:27.326472 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:27.326473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:27.326485 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:27.326550 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:27.326752 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.327314 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.328353 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.328751 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.328846 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:27.328855 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:27.329031 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 03:32:27.329076 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":50,"threads_started":1}
I20260809 03:32:27.329656 29698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987c8d7fb8c1450f96823941ec39ad89" candidate_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30a6bbc7e026479481e557592152f83c" is_pre_election: true
I20260809 03:32:27.329767 29698 raft_consensus.cc:2454] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1c042cf3d2e48d68dcf673b783bcc95 in term 0.
I20260809 03:32:27.329788 29947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987c8d7fb8c1450f96823941ec39ad89" candidate_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63020353344542d3a52f40dd92d21b47" is_pre_election: true
I20260809 03:32:27.329900 29947 raft_consensus.cc:2454] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1c042cf3d2e48d68dcf673b783bcc95 in term 0.
I20260809 03:32:27.330075 30064 leader_election.cc:302] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30a6bbc7e026479481e557592152f83c, c1c042cf3d2e48d68dcf673b783bcc95; no voters:
I20260809 03:32:27.330226 30289 raft_consensus.cc:2781] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:27.330297 30289 raft_consensus.cc:482] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:27.330322 30289 raft_consensus.cc:3037] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:27.331213 30289 raft_consensus.cc:504] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:27.331355 30289 leader_election.cc:288] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [CANDIDATE]: Term 1 election: Requested vote from peers 30a6bbc7e026479481e557592152f83c (127.27.40.129:45969), 63020353344542d3a52f40dd92d21b47 (127.27.40.130:45363)
I20260809 03:32:27.331547 29698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987c8d7fb8c1450f96823941ec39ad89" candidate_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30a6bbc7e026479481e557592152f83c"
I20260809 03:32:27.331645 29698 raft_consensus.cc:3037] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:27.332427 29698 raft_consensus.cc:2454] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1c042cf3d2e48d68dcf673b783bcc95 in term 1.
I20260809 03:32:27.332590 30064 leader_election.cc:302] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30a6bbc7e026479481e557592152f83c, c1c042cf3d2e48d68dcf673b783bcc95; no voters:
I20260809 03:32:27.332572 29947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987c8d7fb8c1450f96823941ec39ad89" candidate_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63020353344542d3a52f40dd92d21b47"
I20260809 03:32:27.332667 29947 raft_consensus.cc:3037] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:27.332752 30289 raft_consensus.cc:2781] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:27.332883 30289 raft_consensus.cc:686] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [term 1 LEADER]: Becoming Leader. State: Replica: c1c042cf3d2e48d68dcf673b783bcc95, State: Running, Role: LEADER
I20260809 03:32:27.332959 30289 consensus_queue.cc:237] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } }
I20260809 03:32:27.333387 30351 tablet_replica.cc:439] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95: TxnStatusTablet state changed. Reason: New leader c1c042cf3d2e48d68dcf673b783bcc95. Latest consensus state: current_term: 1 leader_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } }
I20260809 03:32:27.333516 30351 tablet_replica.cc:442] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:32:27.333618 29947 raft_consensus.cc:2454] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1c042cf3d2e48d68dcf673b783bcc95 in term 1.
I20260809 03:32:27.333734 29455 catalog_manager.cc:5495] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1c042cf3d2e48d68dcf673b783bcc95 (127.27.40.131). New cstate: current_term: 1 leader_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } health_report { overall_health: HEALTHY } } }
I20260809 03:32:27.333874 30353 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 03:32:27.334014 30353 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 03:32:27.345790 30285 consensus_queue.cc:1035] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [LEADER]: Connected to new peer: Peer: permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:27.348224 30285 consensus_queue.cc:1035] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:27.413764 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:27.410927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:39719 (local address 127.0.0.1:42458)
0809 03:32:27.411028 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:27.411042 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:27.411137 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:27.411358 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:27.411359 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:27.411361 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:32:27.411426 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:27.411434 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.412154 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.412155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:27.412734 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:27.412736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.413134 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.413136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:27.413200 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:27.413214 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:32:27.413508 (+ 294us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:32:27.413518 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 03:32:27.413631 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 03:32:27.413684 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 03:32:27.413990 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:27.411035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42458 (local address 127.27.40.131:39719)
0809 03:32:27.411157 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:27.411159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:27.411168 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:27.411229 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:27.411230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:27.411248 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:27.411354 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:27.411518 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.412057 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.412818 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.413047 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.413127 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:27.413307 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:27.413425 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:27.413502 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:27.413635 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 03:32:27.413657 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260809 03:32:27.415098 29698 raft_consensus.cc:1261] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c [term 1 FOLLOWER]: Refusing update from remote peer c1c042cf3d2e48d68dcf673b783bcc95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:27.415205 29947 raft_consensus.cc:1261] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47 [term 1 FOLLOWER]: Refusing update from remote peer c1c042cf3d2e48d68dcf673b783bcc95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:27.415478 30289 consensus_queue.cc:1035] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:27.415674 30289 consensus_queue.cc:1035] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:27.416311 30294 tablet_replica.cc:439] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47: TxnStatusTablet state changed. Reason: New leader c1c042cf3d2e48d68dcf673b783bcc95. Latest consensus state: current_term: 1 leader_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } }
I20260809 03:32:27.416409 30294 tablet_replica.cc:442] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:27.416383 30285 tablet_replica.cc:439] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c: TxnStatusTablet state changed. Reason: New leader c1c042cf3d2e48d68dcf673b783bcc95. Latest consensus state: current_term: 1 leader_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } }
I20260809 03:32:27.416476 30285 tablet_replica.cc:442] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:27.417380 30289 tablet_replica.cc:439] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } }
I20260809 03:32:27.417455 30351 tablet_replica.cc:439] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } }
I20260809 03:32:27.417491 30289 tablet_replica.cc:442] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:32:27.417511 30351 tablet_replica.cc:442] T 987c8d7fb8c1450f96823941ec39ad89 P c1c042cf3d2e48d68dcf673b783bcc95: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:32:27.417467 30358 mvcc.cc:205] Tried to move back new op lower bound from 7316465039010111488 to 7316465038676504576. Current Snapshot: MvccSnapshot[applied={T|T < 7316465039010111488}]
I20260809 03:32:27.417735 30294 tablet_replica.cc:439] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } }
I20260809 03:32:27.417819 30294 tablet_replica.cc:442] T 987c8d7fb8c1450f96823941ec39ad89 P 63020353344542d3a52f40dd92d21b47: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:27.418126 30285 tablet_replica.cc:439] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6bbc7e026479481e557592152f83c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 45969 } } peers { permanent_uuid: "63020353344542d3a52f40dd92d21b47" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45363 } } peers { permanent_uuid: "c1c042cf3d2e48d68dcf673b783bcc95" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39719 } } }
I20260809 03:32:27.418211 30285 tablet_replica.cc:442] T 987c8d7fb8c1450f96823941ec39ad89 P 30a6bbc7e026479481e557592152f83c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:27.423254 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:27.420629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59080 (local address 127.27.40.129:45969)
0809 03:32:27.420682 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:27.420684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:27.420779 (+ 95us) server_negotiation.cc:407] Connection header received
0809 03:32:27.420853 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:27.420854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:27.420874 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:27.420960 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:27.421112 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.421678 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.422432 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.422650 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.422725 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:27.422898 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:27.423010 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:27.423094 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:27.423154 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 03:32:27.423175 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 03:32:27.428865 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:27.425859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:39019 (local address 127.27.40.131:39719)
0809 03:32:27.425916 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:27.425919 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:27.426117 (+ 198us) server_negotiation.cc:407] Connection header received
0809 03:32:27.426217 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:27.426218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:27.426245 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:27.426319 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:27.426508 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.427054 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.428017 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.428238 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.428341 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:27.428557 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:27.428598 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:27.428678 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:27.428757 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 03:32:27.428782 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 03:32:27.428886 30378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:27.425728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:39719 (local address 127.27.40.129:39019)
0809 03:32:27.425979 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:27.425993 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:27.426139 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:27.426338 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:27.426339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:27.426342 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:32:27.426424 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:27.426426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.427155 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.427157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:27.427895 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:27.427898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.428346 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.428347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:27.428427 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:27.428447 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:32:27.428678 (+ 231us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:32:27.428681 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:32:27.428747 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 03:32:27.428800 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":95,"threads_started":1}
I20260809 03:32:27.440392 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:27.437495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:48955 (local address 127.27.40.129:45969)
0809 03:32:27.437580 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:27.437583 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:27.437704 (+ 121us) server_negotiation.cc:407] Connection header received
0809 03:32:27.437797 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:27.437798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:27.437809 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:27.437865 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:27.438063 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.438583 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.439715 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.439891 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.439943 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:27.440194 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:27.440221 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:27.440260 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:27.440322 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 03:32:27.440339 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:32:27.440513 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:27.437307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:45969 (local address 127.27.40.131:48955)
0809 03:32:27.437637 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:27.437650 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:27.437713 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:27.437872 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:27.437873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:27.437875 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:32:27.437962 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:27.437964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.438678 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.438679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:27.439626 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:27.439629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.439973 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.439974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:27.440092 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:27.440108 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:32:27.440267 (+ 159us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:32:27.440269 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 03:32:27.440317 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 03:32:27.440361 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":86,"threads_started":1}
I20260809 03:32:27.449160 29455 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:27.449281 29455 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:27.449957 29455 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f07288df997d4a6f822312d866cae7ae: Sending DeleteTablet for 3 replicas of tablet 09e88905ca304c6988ee69260e035873
I20260809 03:32:27.450206 29671 tablet_service.cc:1514] Processing DeleteTablet for tablet 09e88905ca304c6988ee69260e035873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:27 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:59068
I20260809 03:32:27.450268 29914 tablet_service.cc:1514] Processing DeleteTablet for tablet 09e88905ca304c6988ee69260e035873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:27 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:33858
I20260809 03:32:27.450331 30120 tablet_service.cc:1514] Processing DeleteTablet for tablet 09e88905ca304c6988ee69260e035873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:27 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:42452
I20260809 03:32:27.450809 30389 tablet_replica.cc:330] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47: stopping tablet replica
I20260809 03:32:27.450881 30389 raft_consensus.cc:2229] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:27.450932 30389 raft_consensus.cc:2258] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:27.451197 30392 tablet_replica.cc:330] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c: stopping tablet replica
I20260809 03:32:27.451215 30389 ts_tablet_manager.cc:1905] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:27.451262 30392 raft_consensus.cc:2229] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:32:27.451344 30392 raft_consensus.cc:2258] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:27.451581 30392 ts_tablet_manager.cc:1905] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:27.453284 30392 ts_tablet_manager.cc:1918] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:32:27.453318 30389 ts_tablet_manager.cc:1918] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:32:27.453365 30392 log.cc:1196] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-0/wal/wals/09e88905ca304c6988ee69260e035873
I20260809 03:32:27.453375 30389 log.cc:1196] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/09e88905ca304c6988ee69260e035873
I20260809 03:32:27.453655 30391 tablet_replica.cc:330] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95: stopping tablet replica
I20260809 03:32:27.453691 30392 ts_tablet_manager.cc:1939] T 09e88905ca304c6988ee69260e035873 P 30a6bbc7e026479481e557592152f83c: Deleting consensus metadata
I20260809 03:32:27.453801 30389 ts_tablet_manager.cc:1939] T 09e88905ca304c6988ee69260e035873 P 63020353344542d3a52f40dd92d21b47: Deleting consensus metadata
I20260809 03:32:27.454105 29436 catalog_manager.cc:4841] TS 30a6bbc7e026479481e557592152f83c (127.27.40.129:45969): tablet 09e88905ca304c6988ee69260e035873 (table test-table [id=4487a5a1bc3546469a422ade0d63d298]) successfully deleted
I20260809 03:32:27.454213 29435 catalog_manager.cc:4841] TS 63020353344542d3a52f40dd92d21b47 (127.27.40.130:45363): tablet 09e88905ca304c6988ee69260e035873 (table test-table [id=4487a5a1bc3546469a422ade0d63d298]) successfully deleted
I20260809 03:32:27.454304 30391 raft_consensus.cc:2229] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:27.454385 30391 raft_consensus.cc:2258] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:27.455008 30391 ts_tablet_manager.cc:1905] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:27.456892 30391 ts_tablet_manager.cc:1918] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:32:27.456952 30391 log.cc:1196] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/09e88905ca304c6988ee69260e035873
I20260809 03:32:27.457201 30391 ts_tablet_manager.cc:1939] T 09e88905ca304c6988ee69260e035873 P c1c042cf3d2e48d68dcf673b783bcc95: Deleting consensus metadata
I20260809 03:32:27.457504 29435 catalog_manager.cc:4841] TS c1c042cf3d2e48d68dcf673b783bcc95 (127.27.40.131:39719): tablet 09e88905ca304c6988ee69260e035873 (table test-table [id=4487a5a1bc3546469a422ade0d63d298]) successfully deleted
Aug 09 03:32:27 dist-test-slave-gkw7 krb5kdc[29383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246346, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.40.129@KRBTEST.COM
I20260809 03:32:27.467687 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:27.454436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59082 (local address 127.27.40.129:45969)
0809 03:32:27.454482 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:27.454484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:27.454751 (+ 267us) server_negotiation.cc:407] Connection header received
0809 03:32:27.454804 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:27.454805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:27.454816 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:27.454894 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:27.456078 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.456963 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.458387 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.458782 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.458887 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:27.463813 (+ 4926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:27.463822 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:27.463823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:27.463831 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:27.466133 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:27.466536 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:27.466540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:27.466540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:27.466617 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:27.466867 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:27.466868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:27.466868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:27.467039 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:27.467241 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:27.467477 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 03:32:27.467589 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 03:32:27.468408 29773 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59082
I20260809 03:32:27.478281 30399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:27.469580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38182 (local address 127.27.40.190:41295)
0809 03:32:27.469908 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:27.469910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:27.469917 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:32:27.469944 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:27.469944 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:27.469957 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:27.470020 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:27.470736 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.471514 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.472668 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:27.472969 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:27.473384 (+ 415us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:27.474640 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:27.474641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:27.474642 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:27.474649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:27.476544 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:27.477060 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:27.477061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:27.477062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:27.477130 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:27.477350 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:27.477351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:27.477351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:27.477482 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:27.478040 (+ 558us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:27.478086 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 03:32:27.478177 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":58,"threads_started":1}
W20260809 03:32:27.478801 29455 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38182
I20260809 03:32:27.479528 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 29563
I20260809 03:32:27.492767 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 29808
I20260809 03:32:27.507625 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 30039
I20260809 03:32:27.518991 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 29412
2026-08-09T03:32:27Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3245 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:32:27 dist-test-slave-gkw7 krb5kdc[30403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:32:27 dist-test-slave-gkw7 krb5kdc[30403](info): set up 2 sockets
Aug 09 03:32:27 dist-test-slave-gkw7 krb5kdc[30403](info): commencing operation
krb5kdc: starting...
W20260809 03:32:28.781420 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:32:28 dist-test-slave-gkw7 krb5kdc[30403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:32:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:32:28.908692 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:45839
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:38567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:45839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:28.992205 30428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:28.992413 30428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:28.992435 30428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:28.993499 30428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:28.993587 30428 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 03:32:28.993618 30428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:28.993633 30428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:28.993649 30428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:28.993660 30428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:32:28.995095 30428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:38567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:45839
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:45839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:28.995299 30428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:28.995465 30428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:29.000520 30435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:29.000483 30438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:29.000554 30428 server_base.cc:1034] running on GCE node
W20260809 03:32:29.000471 30436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:29.000982 30428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:29.001281 30428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:29.002449 30428 hybrid_clock.cc:648] HybridClock initialized: now 1786246349002437 us; error 44 us; skew 500 ppm
Aug 09 03:32:28 dist-test-slave-gkw7 krb5kdc[30403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246348, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:29.004995 30428 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:32:29.005440 30428 webserver.cc:460] Webserver started at http://127.27.40.190:36267/ using document root <none> and password file <none>
I20260809 03:32:29.005700 30428 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:29.005748 30428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:29.005841 30428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:29.006803 30428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "d64bc39257074a988e422bb5dd2b28e3"
format_stamp: "Formatted at 2026-08-09 03:32:29 on dist-test-slave-gkw7"
server_key: "f5e956aa55e41e857fc82c32707b166f"
server_key_iv: "238d3573cce3d857a12d68407303eb86"
server_key_version: "encryptionkey@0"
I20260809 03:32:29.007102 30428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "d64bc39257074a988e422bb5dd2b28e3"
format_stamp: "Formatted at 2026-08-09 03:32:29 on dist-test-slave-gkw7"
server_key: "f5e956aa55e41e857fc82c32707b166f"
server_key_iv: "238d3573cce3d857a12d68407303eb86"
server_key_version: "encryptionkey@0"
I20260809 03:32:29.008395 30428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:32:29.009136 30448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:29.009306 30428 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:29.009395 30428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "d64bc39257074a988e422bb5dd2b28e3"
format_stamp: "Formatted at 2026-08-09 03:32:29 on dist-test-slave-gkw7"
server_key: "f5e956aa55e41e857fc82c32707b166f"
server_key_iv: "238d3573cce3d857a12d68407303eb86"
server_key_version: "encryptionkey@0"
I20260809 03:32:29.009464 30428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:29.030007 30428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:29.031982 30428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:29.032150 30428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:29.037108 30428 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:45839
I20260809 03:32:29.037153 30550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:45839 every 8 connection(s)
I20260809 03:32:29.037545 30428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:29.038281 30552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:29.040587 30552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3: Bootstrap starting.
I20260809 03:32:29.041205 30552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:29.041514 30552 log.cc:824] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:29.042284 30552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3: No bootstrap required, opened a new log
I20260809 03:32:29.043718 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 30428
I20260809 03:32:29.043794 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:29.044004 27810 external_mini_cluster.cc:1428] Setting key dfc37c807fce34af55e206185a513c45
I20260809 03:32:29.044804 30552 raft_consensus.cc:348] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64bc39257074a988e422bb5dd2b28e3" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 45839 } }
I20260809 03:32:29.044988 30552 raft_consensus.cc:374] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:29.045025 30552 raft_consensus.cc:729] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d64bc39257074a988e422bb5dd2b28e3, State: Initialized, Role: FOLLOWER
I20260809 03:32:29.045157 30552 consensus_queue.cc:260] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64bc39257074a988e422bb5dd2b28e3" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 45839 } }
I20260809 03:32:29.045253 30552 raft_consensus.cc:388] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:29.045288 30552 raft_consensus.cc:482] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:29.045341 30552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:29.046010 30552 raft_consensus.cc:504] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64bc39257074a988e422bb5dd2b28e3" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 45839 } }
I20260809 03:32:29.046121 30552 leader_election.cc:302] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d64bc39257074a988e422bb5dd2b28e3; no voters:
I20260809 03:32:29.046298 30552 leader_election.cc:288] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:29.046394 30559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:29.046546 30552 sys_catalog.cc:564] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:29.046588 30559 raft_consensus.cc:686] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [term 1 LEADER]: Becoming Leader. State: Replica: d64bc39257074a988e422bb5dd2b28e3, State: Running, Role: LEADER
I20260809 03:32:29.046809 30559 consensus_queue.cc:237] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64bc39257074a988e422bb5dd2b28e3" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 45839 } }
I20260809 03:32:29.047209 30560 sys_catalog.cc:455] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d64bc39257074a988e422bb5dd2b28e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64bc39257074a988e422bb5dd2b28e3" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 45839 } } }
I20260809 03:32:29.047310 30560 sys_catalog.cc:458] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:29.047216 30561 sys_catalog.cc:455] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d64bc39257074a988e422bb5dd2b28e3. Latest consensus state: current_term: 1 leader_uuid: "d64bc39257074a988e422bb5dd2b28e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64bc39257074a988e422bb5dd2b28e3" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 45839 } } }
I20260809 03:32:29.047534 30561 sys_catalog.cc:458] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3 [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:29.047544 30573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:29.048080 30573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:29.050261 30573 catalog_manager.cc:1347] Generated new cluster ID: 694ad75f0ce64bbe91cf10a0deeb3c48
I20260809 03:32:29.050323 30573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 03:32:29 dist-test-slave-gkw7 krb5kdc[30403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:29.055280 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.044615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52312 (local address 127.27.40.190:45839)
0809 03:32:29.044823 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.044833 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.044868 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:32:29.044993 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.045001 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.045060 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.045203 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:29.045983 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.047437 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.048621 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.049152 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.049305 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.051910 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:29.051914 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:29.051915 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:29.051926 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:29.053742 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.054136 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.054139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.054139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.054251 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.054476 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.054476 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.054477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.054725 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:29.054906 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.054971 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.055115 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
I20260809 03:32:29.082204 30573 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:29.082790 30573 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:29.088541 30573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3: Generated new TSK 0
I20260809 03:32:29.088763 30573 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:32:29.116379 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:45839
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:38567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:29.200840 30591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:29.201035 30591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:29.201059 30591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:29.202271 30591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:29.202314 30591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:29.202335 30591 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 03:32:29.202347 30591 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:32:29.202389 30591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:32:29.203810 30591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:38567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.40.190:45839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:29.204046 30591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:29.204269 30591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:29.209733 30605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:29.209717 30598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:29.209719 30596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:29.209949 30591 server_base.cc:1034] running on GCE node
I20260809 03:32:29.210172 30591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:29.210434 30591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:29.211637 30591 hybrid_clock.cc:648] HybridClock initialized: now 1786246349211623 us; error 26 us; skew 500 ppm
Aug 09 03:32:29 dist-test-slave-gkw7 krb5kdc[30403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246349, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:29.214365 30591 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:32:29.214793 30591 webserver.cc:460] Webserver started at http://127.27.40.129:46321/ using document root <none> and password file <none>
I20260809 03:32:29.215044 30591 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:29.215126 30591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:29.215273 30591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:29.216298 30591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "5390d00dd0d74888809954b8045b2d49"
format_stamp: "Formatted at 2026-08-09 03:32:29 on dist-test-slave-gkw7"
server_key: "575ca78ea6e08bd1d09b80cce37680e5"
server_key_iv: "463f6d5875348ce042867f8ff6b612f1"
server_key_version: "encryptionkey@0"
I20260809 03:32:29.216679 30591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "5390d00dd0d74888809954b8045b2d49"
format_stamp: "Formatted at 2026-08-09 03:32:29 on dist-test-slave-gkw7"
server_key: "575ca78ea6e08bd1d09b80cce37680e5"
server_key_iv: "463f6d5875348ce042867f8ff6b612f1"
server_key_version: "encryptionkey@0"
I20260809 03:32:29.218146 30591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:32:29.218874 30619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:29.219044 30591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:32:29.219194 30591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "5390d00dd0d74888809954b8045b2d49"
format_stamp: "Formatted at 2026-08-09 03:32:29 on dist-test-slave-gkw7"
server_key: "575ca78ea6e08bd1d09b80cce37680e5"
server_key_iv: "463f6d5875348ce042867f8ff6b612f1"
server_key_version: "encryptionkey@0"
I20260809 03:32:29.219270 30591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:29.227342 30591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:29.229084 30591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:29.229243 30591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:29.229815 30591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:29.229871 30591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:29.229914 30591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:29.229945 30591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:32:29 dist-test-slave-gkw7 krb5kdc[30403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246349, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:29.237183 30591 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:39783
I20260809 03:32:29.237258 30818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:39783 every 8 connection(s)
I20260809 03:32:29.237592 30591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:32:29.240144 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.230432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:52953 (local address 127.27.40.190:45839)
0809 03:32:29.230490 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.230492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.230699 (+ 207us) server_negotiation.cc:407] Connection header received
0809 03:32:29.231211 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.231213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.231231 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.231295 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:29.232249 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.233088 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.234755 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.235099 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.235171 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.237568 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:29.237569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:29.237570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:29.237578 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:29.238936 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.239293 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.239294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.239294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.239350 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.239570 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.239572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.239572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.239709 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:29.239775 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.239999 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.240077 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 03:32:29.240228 30635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.230373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:45839 (local address 127.27.40.129:52953)
0809 03:32:29.230570 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:29.230605 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:29.231079 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:29.231416 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:29.231422 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:29.231838 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:29.232131 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.232139 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.233194 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.233195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.234662 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.234665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.235190 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.235191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.235343 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.235929 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:29.235939 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:29.237461 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:29.239008 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:29.239014 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:29.239015 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:29.239225 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:29.239406 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:29.239407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:29.239407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:29.239490 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:29.239776 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:29.239782 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:29.239886 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 03:32:29.239966 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 03:32:29.240079 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":115,"threads_started":1}
I20260809 03:32:29.240747 30819 heartbeater.cc:347] Connected to a master server at 127.27.40.190:45839
I20260809 03:32:29.240847 30819 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:29.241032 30819 heartbeater.cc:511] Master 127.27.40.190:45839 requested a full tablet report, sending...
I20260809 03:32:29.241592 30475 ts_manager.cc:194] Registered new tserver with Master: 5390d00dd0d74888809954b8045b2d49 (127.27.40.129:39783)
I20260809 03:32:29.241997 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 30591
I20260809 03:32:29.242105 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:32:29.242273 27810 external_mini_cluster.cc:1428] Setting key 7d768da48ccaa1fbfab1aae6c95caacf
I20260809 03:32:29.242552 30475 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:52953
I20260809 03:32:29.247866 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.241711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:45839 (local address 127.27.40.129:52987)
0809 03:32:29.241885 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:29.241896 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:29.241960 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:29.242124 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:29.242126 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:29.242317 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:29.242388 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.242389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.243200 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.243200 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.243700 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.243701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.244081 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.244082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.244159 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.244618 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:29.244623 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:29.244938 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:29.246724 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:29.246725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:29.246725 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:29.246894 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:29.247154 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:29.247154 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:29.247155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:29.247212 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:29.247483 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:29.247483 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:29.247576 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 03:32:29.247673 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 03:32:29.247796 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260809 03:32:29.247964 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.241759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:52987 (local address 127.27.40.190:45839)
0809 03:32:29.241823 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.241826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.241964 (+ 138us) server_negotiation.cc:407] Connection header received
0809 03:32:29.242009 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.242010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.242020 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.242079 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:29.242469 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.243098 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.243783 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.243991 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.244080 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.245049 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:29.245051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:29.245051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:29.245059 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:29.246618 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.246997 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.246998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.246998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.247061 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.247276 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.247277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.247277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.247417 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:29.247485 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.247727 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.247824 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:32:29.268688 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:45839
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:38567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:29.355621 30830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:29.355832 30830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:29.355857 30830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:29.356967 30830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:29.357009 30830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:29.357030 30830 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 03:32:29.357077 30830 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:32:29.357105 30830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:32:29.358608 30830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:38567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.40.190:45839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:29.358867 30830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:29.359113 30830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:29.363978 30842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:29.364007 30838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:29.364013 30837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:29.364493 30830 server_base.cc:1034] running on GCE node
I20260809 03:32:29.364705 30830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:29.364948 30830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:29.366111 30830 hybrid_clock.cc:648] HybridClock initialized: now 1786246349366101 us; error 43 us; skew 500 ppm
Aug 09 03:32:29 dist-test-slave-gkw7 krb5kdc[30403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246349, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:29.368646 30830 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:32:29.369076 30830 webserver.cc:460] Webserver started at http://127.27.40.130:38551/ using document root <none> and password file <none>
I20260809 03:32:29.369323 30830 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:29.369391 30830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:29.369590 30830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:29.370831 30830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "f6584f38965f4556b39abcc6c0e3eb5f"
format_stamp: "Formatted at 2026-08-09 03:32:29 on dist-test-slave-gkw7"
server_key: "94a1319721ff7f70273f9487aa7ed814"
server_key_iv: "418bcb704bcfa8ef5ec4745542d0e984"
server_key_version: "encryptionkey@0"
I20260809 03:32:29.371201 30830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "f6584f38965f4556b39abcc6c0e3eb5f"
format_stamp: "Formatted at 2026-08-09 03:32:29 on dist-test-slave-gkw7"
server_key: "94a1319721ff7f70273f9487aa7ed814"
server_key_iv: "418bcb704bcfa8ef5ec4745542d0e984"
server_key_version: "encryptionkey@0"
I20260809 03:32:29.372511 30830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:32:29.373278 30855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:29.373482 30830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:32:29.373624 30830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "f6584f38965f4556b39abcc6c0e3eb5f"
format_stamp: "Formatted at 2026-08-09 03:32:29 on dist-test-slave-gkw7"
server_key: "94a1319721ff7f70273f9487aa7ed814"
server_key_iv: "418bcb704bcfa8ef5ec4745542d0e984"
server_key_version: "encryptionkey@0"
I20260809 03:32:29.373703 30830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:29.397127 30830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:29.399694 30830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:29.399878 30830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:29.400434 30830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:29.400499 30830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:29.400550 30830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:29.400578 30830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:29.407641 30830 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:35553
I20260809 03:32:29.407721 31040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:35553 every 8 connection(s)
Aug 09 03:32:29 dist-test-slave-gkw7 krb5kdc[30403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246349, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:29.408078 30830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:32:29.411196 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.401012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:34381 (local address 127.27.40.190:45839)
0809 03:32:29.401055 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.401058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.401308 (+ 250us) server_negotiation.cc:407] Connection header received
0809 03:32:29.401954 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.401955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.401970 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.402032 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:29.402902 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.403786 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.405340 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.405680 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.405762 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.408565 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:29.408567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:29.408567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:29.408574 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:29.410005 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.410363 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.410364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.410364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.410414 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.410678 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.410679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.410679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.410779 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:29.410829 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.411057 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.411115 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:32:29.411345 30872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.400999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:45839 (local address 127.27.40.130:34381)
0809 03:32:29.401182 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:29.401217 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:29.401811 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:29.402106 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:29.402109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:29.402481 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:29.402781 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.402787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.403889 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.403890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.405236 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.405239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.405779 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.405780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.405944 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.406481 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:29.406493 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:29.408451 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:29.410074 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:29.410079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:29.410081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:29.410295 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:29.410496 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:29.410498 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:29.410498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:29.410614 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:29.410836 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:29.410840 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:29.410950 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 03:32:29.411031 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 03:32:29.411156 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260809 03:32:29.411958 31043 heartbeater.cc:347] Connected to a master server at 127.27.40.190:45839
I20260809 03:32:29.412106 31043 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:29.412294 31043 heartbeater.cc:511] Master 127.27.40.190:45839 requested a full tablet report, sending...
I20260809 03:32:29.412741 30477 ts_manager.cc:194] Registered new tserver with Master: f6584f38965f4556b39abcc6c0e3eb5f (127.27.40.130:35553)
I20260809 03:32:29.413280 30477 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:34381
I20260809 03:32:29.413981 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 30830
I20260809 03:32:29.414083 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:32:29.414268 27810 external_mini_cluster.cc:1428] Setting key be8b1bbd0bd5555a0d15bead8054f23e
I20260809 03:32:29.419402 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.413221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:48539 (local address 127.27.40.190:45839)
0809 03:32:29.413257 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.413259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.413401 (+ 142us) server_negotiation.cc:407] Connection header received
0809 03:32:29.413493 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.413495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.413506 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.413601 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:29.413988 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.414673 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.415621 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.415793 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.415849 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.416905 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:29.416906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:29.416907 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:29.416913 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:29.418197 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.418564 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.418566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.418566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.418626 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.418844 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.418845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.418845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.418995 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:29.419072 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.419235 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.419312 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 03:32:29.419416 31055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.413195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:45839 (local address 127.27.40.130:48539)
0809 03:32:29.413319 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:29.413347 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:29.413441 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:29.413613 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:29.413614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:29.413807 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:29.413886 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.413888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.414800 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.414801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.415535 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.415538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.415869 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.415870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.415951 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.416367 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:29.416372 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:29.416795 (+ 423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:29.418296 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:29.418297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:29.418297 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:29.418480 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:29.418693 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:29.418694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:29.418694 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:29.418754 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:29.419076 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:29.419077 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:29.419123 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 03:32:29.419208 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 03:32:29.419312 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:32:29.442428 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:45839
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:38567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:29.529924 31060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:29.530256 31060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:29.530309 31060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:29.532191 31060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:29.532254 31060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:29.532295 31060 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 03:32:29.532330 31060 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:32:29.532372 31060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:32:29.534888 31060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:38567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.40.190:45839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:29.535322 31060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:29.535601 31060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:29.542258 31065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:29.542265 31066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:29.542358 31070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:29.542780 31060 server_base.cc:1034] running on GCE node
I20260809 03:32:29.542996 31060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:29.543253 31060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:29.544415 31060 hybrid_clock.cc:648] HybridClock initialized: now 1786246349544406 us; error 38 us; skew 500 ppm
Aug 09 03:32:29 dist-test-slave-gkw7 krb5kdc[30403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246349, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:29.547298 31060 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:32:29.547856 31060 webserver.cc:460] Webserver started at http://127.27.40.131:37217/ using document root <none> and password file <none>
I20260809 03:32:29.548116 31060 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:29.548189 31060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:29.548345 31060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:29.549446 31060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca"
format_stamp: "Formatted at 2026-08-09 03:32:29 on dist-test-slave-gkw7"
server_key: "59e872bebd42e6aad702ac9167f91aa0"
server_key_iv: "af6e906f287cadc025b8e016fee9279c"
server_key_version: "encryptionkey@0"
I20260809 03:32:29.549888 31060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca"
format_stamp: "Formatted at 2026-08-09 03:32:29 on dist-test-slave-gkw7"
server_key: "59e872bebd42e6aad702ac9167f91aa0"
server_key_iv: "af6e906f287cadc025b8e016fee9279c"
server_key_version: "encryptionkey@0"
I20260809 03:32:29.551258 31060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:29.552067 31084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:29.552246 31060 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:29.552356 31060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca"
format_stamp: "Formatted at 2026-08-09 03:32:29 on dist-test-slave-gkw7"
server_key: "59e872bebd42e6aad702ac9167f91aa0"
server_key_iv: "af6e906f287cadc025b8e016fee9279c"
server_key_version: "encryptionkey@0"
I20260809 03:32:29.552430 31060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:29.564729 31060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:29.567497 31060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:29.567687 31060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:29.568284 31060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:29.568368 31060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:29.568421 31060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:29.568449 31060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:32:29 dist-test-slave-gkw7 krb5kdc[30403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246349, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:29.580762 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.568814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:33709 (local address 127.27.40.190:45839)
0809 03:32:29.568891 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.568895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.569139 (+ 244us) server_negotiation.cc:407] Connection header received
0809 03:32:29.569686 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.569688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.569704 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.569800 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:29.570467 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.571417 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.573056 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.573506 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.573644 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.575901 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:29.575902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:29.575903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:29.575913 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:29.578705 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.579257 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.579259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.579259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.579359 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.579775 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.579776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.579776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.579989 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:29.580106 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.580480 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.580615 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:32:29.580780 31096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.568782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:45839 (local address 127.27.40.131:33709)
0809 03:32:29.568972 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:29.569012 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:29.569523 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:29.569868 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:29.569871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:29.570154 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:29.570367 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.570372 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.571541 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.571542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.572923 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.572926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.573652 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.573653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.573758 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.574303 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:29.574311 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:29.575816 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:29.578828 (+ 3012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:29.578835 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:29.578838 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:29.579162 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:29.579506 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:29.579507 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:29.579508 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:29.579673 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:29.580098 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:29.580105 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:29.580314 (+ 209us) client_negotiation.cc:769] Sending connection context
0809 03:32:29.580420 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 03:32:29.580555 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":107,"threads_started":1}
I20260809 03:32:29.581146 31060 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:45883
I20260809 03:32:29.581240 31267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:45883 every 8 connection(s)
I20260809 03:32:29.581723 31060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:32:29.582765 31270 heartbeater.cc:347] Connected to a master server at 127.27.40.190:45839
I20260809 03:32:29.582962 31270 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:29.583138 31270 heartbeater.cc:511] Master 127.27.40.190:45839 requested a full tablet report, sending...
I20260809 03:32:29.583482 30477 ts_manager.cc:194] Registered new tserver with Master: 9cb53b4ad9744b81a4ab8ec6cb2f8cca (127.27.40.131:45883)
I20260809 03:32:29.583920 30477 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:33709
I20260809 03:32:29.589262 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.582956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:43457 (local address 127.27.40.190:45839)
0809 03:32:29.583008 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.583010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.583190 (+ 180us) server_negotiation.cc:407] Connection header received
0809 03:32:29.583262 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.583263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.583274 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.583330 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:29.583867 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.584491 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.585240 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.585450 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.585560 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.586564 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:29.586566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:29.586566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:29.586573 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:29.588022 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.588397 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.588398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.588399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.588485 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.588770 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.588771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.588771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.588880 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:29.588933 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.589118 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.589200 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 03:32:29.589270 31276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.582891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:45839 (local address 127.27.40.131:43457)
0809 03:32:29.583112 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:29.583122 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:29.583203 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:29.583335 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:29.583336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:29.583640 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:29.583784 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.583786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.584619 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.584620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.585130 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.585132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.585561 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.585563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.585641 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.586128 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:29.586133 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:29.586450 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:29.588128 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:29.588129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:29.588129 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:29.588313 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:29.588569 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:29.588598 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:29.588598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:29.588690 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:29.588946 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:29.588947 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:29.589003 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 03:32:29.589067 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 03:32:29.589192 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":82,"threads_started":1}
I20260809 03:32:29.590569 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 31060
I20260809 03:32:29.590684 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:32:29.590863 27810 external_mini_cluster.cc:1428] Setting key 73c258949768cc80fd2886bb4dd3308a
I20260809 03:32:29.591723 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 03:32:29 dist-test-slave-gkw7 krb5kdc[30403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 03:32:29 dist-test-slave-gkw7 krb5kdc[30403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:29.617208 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.609980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52326 (local address 127.27.40.190:45839)
0809 03:32:29.610051 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.610053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.610063 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:32:29.610139 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.610139 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.610158 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.610226 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:29.610839 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.611326 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.612105 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.612297 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.612372 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.614818 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:29.614819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:29.614820 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:29.614828 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:29.616051 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.616397 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.616398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.616398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.616455 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.616679 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.616680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.616680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.616794 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:29.616905 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.617016 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.617099 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 03:32:29.618885 30477 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:32:29.620436 30477 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:29.627079 31289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.622826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:39783 (local address 127.0.0.1:35366)
0809 03:32:29.623025 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:29.623070 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:29.623140 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:29.623802 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:29.623803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:29.623805 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:29.623851 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.623853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.625101 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.625102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.626060 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.626061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.626737 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.626738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.626818 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.626832 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 03:32:29.626933 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 03:32:29.626989 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":6,"thread_start_us":63,"threads_started":1}
I20260809 03:32:29.627089 31292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.622953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35366 (local address 127.27.40.129:39783)
0809 03:32:29.623577 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.623582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.623591 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:29.623630 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.623632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.623647 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.623737 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:29.623939 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.625012 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.626143 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.626625 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.626732 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.626797 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.626954 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.626993 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":105,"threads_started":1}
I20260809 03:32:29.627193 31291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.622859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:35553 (local address 127.0.0.1:35408)
0809 03:32:29.623140 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:29.623150 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:29.623193 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:29.623473 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:29.623476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:29.623478 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:29.623560 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.623562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.624556 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.624557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.625691 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.625694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.626899 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.626900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.626970 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.626974 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:29.627036 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 03:32:29.627113 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":94,"threads_started":1}
I20260809 03:32:29.627436 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.622974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35408 (local address 127.27.40.130:35553)
0809 03:32:29.623308 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.623312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.623320 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:32:29.623351 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.623352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.623365 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.623454 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:29.623650 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.624467 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.625771 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.626732 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.626804 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.626862 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.627277 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.627323 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":77,"threads_started":1}
I20260809 03:32:29.627743 31294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.623038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:45883 (local address 127.0.0.1:54182)
0809 03:32:29.623198 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:29.623209 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:29.623265 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:29.623715 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:29.623716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:29.623717 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:29.623771 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.623773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.625106 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.625107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.626433 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.626437 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.627507 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.627508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.627589 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.627594 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:32:29.627631 (+ 37us) client_negotiation.cc:240] Negotiation successful
0809 03:32:29.627669 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":2,"thread_start_us":111,"threads_started":1}
I20260809 03:32:29.628237 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.623289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54182 (local address 127.27.40.131:45883)
0809 03:32:29.623526 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.623530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.623541 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:32:29.623592 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.623594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.623609 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.623846 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:29.623913 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.624995 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.626575 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.627395 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.627963 (+ 568us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.628039 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.628068 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.628118 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":81,"threads_started":1}
I20260809 03:32:29.628232 30698 tablet_service.cc:1467] Processing CreateTablet for tablet c6716a852ff847ccb2f2c3a3ff937458 (DEFAULT_TABLE table=test-table [id=9d2789f3541d436798d56c63a93654e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:29.628297 30932 tablet_service.cc:1467] Processing CreateTablet for tablet c6716a852ff847ccb2f2c3a3ff937458 (DEFAULT_TABLE table=test-table [id=9d2789f3541d436798d56c63a93654e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:29.628643 30932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6716a852ff847ccb2f2c3a3ff937458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:29.628643 30698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6716a852ff847ccb2f2c3a3ff937458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:29.630185 31160 tablet_service.cc:1467] Processing CreateTablet for tablet c6716a852ff847ccb2f2c3a3ff937458 (DEFAULT_TABLE table=test-table [id=9d2789f3541d436798d56c63a93654e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:29.630533 31160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6716a852ff847ccb2f2c3a3ff937458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:29.632833 31298 tablet_bootstrap.cc:492] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49: Bootstrap starting.
I20260809 03:32:29.633632 31298 tablet_bootstrap.cc:654] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:29.633677 31299 tablet_bootstrap.cc:492] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Bootstrap starting.
I20260809 03:32:29.633671 31296 tablet_bootstrap.cc:492] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f: Bootstrap starting.
I20260809 03:32:29.634292 31298 log.cc:824] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:29.634411 31296 tablet_bootstrap.cc:654] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:29.634799 31299 tablet_bootstrap.cc:654] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:29.634927 31296 log.cc:824] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:29.635167 31299 log.cc:824] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:29.635196 31298 tablet_bootstrap.cc:492] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49: No bootstrap required, opened a new log
I20260809 03:32:29.635270 31298 ts_tablet_manager.cc:1397] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 03:32:29.635666 31296 tablet_bootstrap.cc:492] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f: No bootstrap required, opened a new log
I20260809 03:32:29.635768 31296 ts_tablet_manager.cc:1397] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:32:29.636943 31298 raft_consensus.cc:348] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } }
I20260809 03:32:29.637080 31298 raft_consensus.cc:374] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:29.637111 31298 raft_consensus.cc:729] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5390d00dd0d74888809954b8045b2d49, State: Initialized, Role: FOLLOWER
I20260809 03:32:29.637244 31298 consensus_queue.cc:260] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } }
I20260809 03:32:29.637593 31298 ts_tablet_manager.cc:1428] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 03:32:29.637637 30819 heartbeater.cc:503] Master 127.27.40.190:45839 was elected leader, sending a full tablet report...
I20260809 03:32:29.637611 31296 raft_consensus.cc:348] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } }
I20260809 03:32:29.637786 31296 raft_consensus.cc:374] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:29.637830 31296 raft_consensus.cc:729] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6584f38965f4556b39abcc6c0e3eb5f, State: Initialized, Role: FOLLOWER
I20260809 03:32:29.637951 31296 consensus_queue.cc:260] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } }
I20260809 03:32:29.639591 31296 ts_tablet_manager.cc:1428] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 03:32:29.639678 31043 heartbeater.cc:503] Master 127.27.40.190:45839 was elected leader, sending a full tablet report...
I20260809 03:32:29.640304 31299 tablet_bootstrap.cc:492] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: No bootstrap required, opened a new log
I20260809 03:32:29.640380 31299 ts_tablet_manager.cc:1397] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260809 03:32:29.642336 31299 raft_consensus.cc:348] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } }
I20260809 03:32:29.642488 31299 raft_consensus.cc:374] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:29.642534 31299 raft_consensus.cc:729] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb53b4ad9744b81a4ab8ec6cb2f8cca, State: Initialized, Role: FOLLOWER
I20260809 03:32:29.642652 31299 consensus_queue.cc:260] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } }
I20260809 03:32:29.643028 31299 ts_tablet_manager.cc:1428] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:32:29.643123 31270 heartbeater.cc:503] Master 127.27.40.190:45839 was elected leader, sending a full tablet report...
W20260809 03:32:29.658687 31045 tablet.cc:2369] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:32:29.738602 30820 tablet.cc:2369] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:32:29.788836 31308 raft_consensus.cc:482] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:29.788976 31308 raft_consensus.cc:504] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } }
I20260809 03:32:29.789326 31308 leader_election.cc:288] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5390d00dd0d74888809954b8045b2d49 (127.27.40.129:39783), 9cb53b4ad9744b81a4ab8ec6cb2f8cca (127.27.40.131:45883)
I20260809 03:32:29.793440 31292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.789705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:38625 (local address 127.27.40.129:39783)
0809 03:32:29.789773 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.789777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.789791 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:32:29.789871 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.789872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.789883 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.789964 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:29.790214 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.790992 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.792485 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.793012 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.793087 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.793095 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.793335 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.793371 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 03:32:29.793440 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.789967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:40669 (local address 127.27.40.131:45883)
0809 03:32:29.790017 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.790019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.790198 (+ 179us) server_negotiation.cc:407] Connection header received
0809 03:32:29.790227 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.790227 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.790237 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.790289 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:29.790432 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.791098 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.792478 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.793012 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.793087 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.793095 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.793329 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.793368 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 03:32:29.793468 30872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.789662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:39783 (local address 127.27.40.130:38625)
0809 03:32:29.789741 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:29.789756 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:29.789806 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:29.789972 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:29.789973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:29.789975 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:29.790058 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.790061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.791072 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.791073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.792388 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.792391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.793149 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.793150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.793239 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.793242 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:32:29.793312 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 03:32:29.793350 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":15}
I20260809 03:32:29.793598 31310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.789822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:45883 (local address 127.27.40.130:40669)
0809 03:32:29.790069 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:29.790080 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:29.790136 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:29.790306 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:29.790306 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:29.790308 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:29.790362 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.790364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.791184 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.791184 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.792389 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.792391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.793131 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.793133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.793216 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.793222 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:32:29.793303 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 03:32:29.793350 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":38,"thread_start_us":175,"threads_started":1}
I20260809 03:32:29.793984 31187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6716a852ff847ccb2f2c3a3ff937458" candidate_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" is_pre_election: true
I20260809 03:32:29.793993 30747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6716a852ff847ccb2f2c3a3ff937458" candidate_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5390d00dd0d74888809954b8045b2d49" is_pre_election: true
I20260809 03:32:29.794114 31187 raft_consensus.cc:2454] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6584f38965f4556b39abcc6c0e3eb5f in term 0.
I20260809 03:32:29.794164 30747 raft_consensus.cc:2454] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6584f38965f4556b39abcc6c0e3eb5f in term 0.
I20260809 03:32:29.794476 30861 leader_election.cc:302] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5390d00dd0d74888809954b8045b2d49, f6584f38965f4556b39abcc6c0e3eb5f; no voters:
I20260809 03:32:29.794613 31308 raft_consensus.cc:2781] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:29.794705 31308 raft_consensus.cc:482] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:29.794739 31308 raft_consensus.cc:3037] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:29.795809 31308 raft_consensus.cc:504] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } }
I20260809 03:32:29.795962 31308 leader_election.cc:288] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [CANDIDATE]: Term 1 election: Requested vote from peers 5390d00dd0d74888809954b8045b2d49 (127.27.40.129:39783), 9cb53b4ad9744b81a4ab8ec6cb2f8cca (127.27.40.131:45883)
I20260809 03:32:29.796186 31187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6716a852ff847ccb2f2c3a3ff937458" candidate_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca"
I20260809 03:32:29.796258 31187 raft_consensus.cc:3037] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:29.797175 31187 raft_consensus.cc:2454] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6584f38965f4556b39abcc6c0e3eb5f in term 1.
I20260809 03:32:29.797309 30747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6716a852ff847ccb2f2c3a3ff937458" candidate_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5390d00dd0d74888809954b8045b2d49"
I20260809 03:32:29.797384 30861 leader_election.cc:302] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb53b4ad9744b81a4ab8ec6cb2f8cca, f6584f38965f4556b39abcc6c0e3eb5f; no voters:
I20260809 03:32:29.797394 30747 raft_consensus.cc:3037] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:29.797482 31308 raft_consensus.cc:2781] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:29.797694 31308 raft_consensus.cc:686] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 LEADER]: Becoming Leader. State: Replica: f6584f38965f4556b39abcc6c0e3eb5f, State: Running, Role: LEADER
I20260809 03:32:29.797799 31308 consensus_queue.cc:237] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } }
I20260809 03:32:29.798347 30747 raft_consensus.cc:2454] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6584f38965f4556b39abcc6c0e3eb5f in term 1.
I20260809 03:32:29.798949 30477 catalog_manager.cc:5495] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f reported cstate change: term changed from 0 to 1, leader changed from <none> to f6584f38965f4556b39abcc6c0e3eb5f (127.27.40.130). New cstate: current_term: 1 leader_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } health_report { overall_health: HEALTHY } } }
Aug 09 03:32:29 dist-test-slave-gkw7 krb5kdc[30403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246348, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:29.825341 31323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.818279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:45839 (local address 127.0.0.1:52338)
0809 03:32:29.818504 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:29.818515 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:29.818579 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:29.818705 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:29.818706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:29.819040 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:29.819089 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.819092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.819724 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.819725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.820261 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.820263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.820590 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.820591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.820634 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.821210 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:29.821215 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:29.822751 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:29.824370 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:29.824378 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:29.824380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:29.824581 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:29.824779 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:29.824779 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:29.824780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:29.824838 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:29.825046 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:29.825053 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:29.825094 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 03:32:29.825154 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 03:32:29.825246 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":128,"threads_started":1}
I20260809 03:32:29.825388 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.818323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52338 (local address 127.27.40.190:45839)
0809 03:32:29.818394 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.818397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.818579 (+ 182us) server_negotiation.cc:407] Connection header received
0809 03:32:29.818622 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.818623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.818633 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.818685 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:29.819157 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.819657 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.820340 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.820523 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.820577 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.822831 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:29.822832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:29.822832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:29.822838 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:29.824292 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.824667 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.824668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.824669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.824720 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:29.824893 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:29.824894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:29.824894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:29.824993 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:29.825042 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.825181 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.825291 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":38}
I20260809 03:32:29.827502 30477 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:52338:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 03:32:29.827733 30477 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:29.829309 30698 tablet_service.cc:1467] Processing CreateTablet for tablet 0371ce715c794dcb99e357c5b3f2c19d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b161ba4b673045fea03d99d01626162f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:32:29.829339 31160 tablet_service.cc:1467] Processing CreateTablet for tablet 0371ce715c794dcb99e357c5b3f2c19d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b161ba4b673045fea03d99d01626162f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:32:29.829371 30932 tablet_service.cc:1467] Processing CreateTablet for tablet 0371ce715c794dcb99e357c5b3f2c19d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b161ba4b673045fea03d99d01626162f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:32:29.829504 30698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0371ce715c794dcb99e357c5b3f2c19d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:29.829519 30932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0371ce715c794dcb99e357c5b3f2c19d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:29.829519 31160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0371ce715c794dcb99e357c5b3f2c19d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:29.830969 31298 tablet_bootstrap.cc:492] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49: Bootstrap starting.
I20260809 03:32:29.831390 31296 tablet_bootstrap.cc:492] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f: Bootstrap starting.
I20260809 03:32:29.831611 31298 tablet_bootstrap.cc:654] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:29.831975 31296 tablet_bootstrap.cc:654] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:29.832541 31299 tablet_bootstrap.cc:492] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Bootstrap starting.
I20260809 03:32:29.832674 31298 tablet_bootstrap.cc:492] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49: No bootstrap required, opened a new log
I20260809 03:32:29.832722 31298 ts_tablet_manager.cc:1397] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:32:29.832878 31298 raft_consensus.cc:348] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.832930 31298 raft_consensus.cc:374] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:29.832943 31298 raft_consensus.cc:729] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5390d00dd0d74888809954b8045b2d49, State: Initialized, Role: FOLLOWER
I20260809 03:32:29.832978 31298 consensus_queue.cc:260] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.833094 31298 ts_tablet_manager.cc:1428] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:29.833155 31307 tablet_replica.cc:439] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } }
I20260809 03:32:29.833225 31307 tablet_replica.cc:442] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:29.833225 31299 tablet_bootstrap.cc:654] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Neither blocks nor log segments found. Creating new log.
W20260809 03:32:29.833684 31275 tablet.cc:2369] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:32:29.834029 31299 tablet_bootstrap.cc:492] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: No bootstrap required, opened a new log
I20260809 03:32:29.834079 31299 ts_tablet_manager.cc:1397] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 03:32:29.834233 31296 tablet_bootstrap.cc:492] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f: No bootstrap required, opened a new log
I20260809 03:32:29.834273 31296 ts_tablet_manager.cc:1397] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 03:32:29.834338 31299 raft_consensus.cc:348] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.834412 31299 raft_consensus.cc:374] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:29.834434 31299 raft_consensus.cc:729] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb53b4ad9744b81a4ab8ec6cb2f8cca, State: Initialized, Role: FOLLOWER
I20260809 03:32:29.834477 31299 consensus_queue.cc:260] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.834582 31299 ts_tablet_manager.cc:1428] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:29.834508 31296 raft_consensus.cc:348] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.834614 31296 raft_consensus.cc:374] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:29.834651 31296 raft_consensus.cc:729] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6584f38965f4556b39abcc6c0e3eb5f, State: Initialized, Role: FOLLOWER
I20260809 03:32:29.834654 31309 tablet_replica.cc:439] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } }
I20260809 03:32:29.834702 31296 consensus_queue.cc:260] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.834754 31309 tablet_replica.cc:442] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:29.834851 31296 ts_tablet_manager.cc:1428] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 03:32:29.834903 31308 tablet_replica.cc:439] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } }
I20260809 03:32:29.834995 31308 tablet_replica.cc:442] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:29.916236 31308 raft_consensus.cc:482] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:29.916354 31308 raft_consensus.cc:504] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.916482 31308 leader_election.cc:288] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5390d00dd0d74888809954b8045b2d49 (127.27.40.129:39783), 9cb53b4ad9744b81a4ab8ec6cb2f8cca (127.27.40.131:45883)
I20260809 03:32:29.916772 30747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0371ce715c794dcb99e357c5b3f2c19d" candidate_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5390d00dd0d74888809954b8045b2d49" is_pre_election: true
I20260809 03:32:29.916885 30747 raft_consensus.cc:2454] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6584f38965f4556b39abcc6c0e3eb5f in term 0.
I20260809 03:32:29.916846 31187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0371ce715c794dcb99e357c5b3f2c19d" candidate_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" is_pre_election: true
I20260809 03:32:29.917016 31187 raft_consensus.cc:2454] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6584f38965f4556b39abcc6c0e3eb5f in term 0.
I20260809 03:32:29.917068 30861 leader_election.cc:302] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5390d00dd0d74888809954b8045b2d49, f6584f38965f4556b39abcc6c0e3eb5f; no voters:
I20260809 03:32:29.917202 31308 raft_consensus.cc:2781] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:29.917282 31308 raft_consensus.cc:482] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:29.917311 31308 raft_consensus.cc:3037] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:29.918049 31308 raft_consensus.cc:504] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.918197 31308 leader_election.cc:288] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [CANDIDATE]: Term 1 election: Requested vote from peers 5390d00dd0d74888809954b8045b2d49 (127.27.40.129:39783), 9cb53b4ad9744b81a4ab8ec6cb2f8cca (127.27.40.131:45883)
I20260809 03:32:29.918397 30747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0371ce715c794dcb99e357c5b3f2c19d" candidate_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5390d00dd0d74888809954b8045b2d49"
I20260809 03:32:29.918431 31187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0371ce715c794dcb99e357c5b3f2c19d" candidate_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca"
I20260809 03:32:29.918489 31187 raft_consensus.cc:3037] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:29.918502 30747 raft_consensus.cc:3037] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:29.919149 30747 raft_consensus.cc:2454] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6584f38965f4556b39abcc6c0e3eb5f in term 1.
I20260809 03:32:29.919282 31187 raft_consensus.cc:2454] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6584f38965f4556b39abcc6c0e3eb5f in term 1.
I20260809 03:32:29.919386 30861 leader_election.cc:302] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5390d00dd0d74888809954b8045b2d49, f6584f38965f4556b39abcc6c0e3eb5f; no voters:
I20260809 03:32:29.919516 31308 raft_consensus.cc:2781] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:29.919584 31308 raft_consensus.cc:686] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [term 1 LEADER]: Becoming Leader. State: Replica: f6584f38965f4556b39abcc6c0e3eb5f, State: Running, Role: LEADER
I20260809 03:32:29.919623 31308 consensus_queue.cc:237] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.919955 31312 tablet_replica.cc:439] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f: TxnStatusTablet state changed. Reason: New leader f6584f38965f4556b39abcc6c0e3eb5f. Latest consensus state: current_term: 1 leader_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } }
I20260809 03:32:29.920051 31312 tablet_replica.cc:442] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:32:29.920264 31328 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 03:32:29.920413 31328 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 03:32:29.920373 30477 catalog_manager.cc:5495] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f reported cstate change: term changed from 0 to 1, leader changed from <none> to f6584f38965f4556b39abcc6c0e3eb5f (127.27.40.130). New cstate: current_term: 1 leader_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } health_report { overall_health: UNKNOWN } } }
I20260809 03:32:29.929088 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.926216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35422 (local address 127.27.40.130:35553)
0809 03:32:29.926283 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.926285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.926292 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:32:29.926392 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.926393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.926408 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.926467 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:29.926621 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.927141 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.928313 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.928487 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.928531 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.928774 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:29.928859 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:29.928921 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.929022 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.929038 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 03:32:29.929170 31323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.926170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:35553 (local address 127.0.0.1:35422)
0809 03:32:29.926251 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:29.926263 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:29.926311 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:29.926480 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:29.926481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:29.926483 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:32:29.926549 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.926552 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.927228 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.927230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.928223 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.928226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.928558 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.928560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.928654 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.928675 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:32:29.928934 (+ 259us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:32:29.928947 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 03:32:29.929021 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 03:32:29.929070 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 03:32:29.931283 31187 raft_consensus.cc:1261] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 FOLLOWER]: Refusing update from remote peer f6584f38965f4556b39abcc6c0e3eb5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:29.931352 30747 raft_consensus.cc:1261] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Refusing update from remote peer f6584f38965f4556b39abcc6c0e3eb5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:29.931665 31312 consensus_queue.cc:1035] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:29.931780 31308 consensus_queue.cc:1035] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:29.933090 31307 tablet_replica.cc:439] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49: TxnStatusTablet state changed. Reason: New leader f6584f38965f4556b39abcc6c0e3eb5f. Latest consensus state: current_term: 1 leader_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } }
I20260809 03:32:29.933182 31307 tablet_replica.cc:442] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:29.933113 31309 tablet_replica.cc:439] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: TxnStatusTablet state changed. Reason: New leader f6584f38965f4556b39abcc6c0e3eb5f. Latest consensus state: current_term: 1 leader_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } }
I20260809 03:32:29.933212 31309 tablet_replica.cc:442] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:29.934748 31308 tablet_replica.cc:439] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } }
I20260809 03:32:29.934854 31308 tablet_replica.cc:442] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:32:29.934907 31307 tablet_replica.cc:439] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } }
I20260809 03:32:29.934989 31307 tablet_replica.cc:442] T 0371ce715c794dcb99e357c5b3f2c19d P 5390d00dd0d74888809954b8045b2d49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:29.935782 31312 tablet_replica.cc:439] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } }
I20260809 03:32:29.935884 31312 tablet_replica.cc:442] T 0371ce715c794dcb99e357c5b3f2c19d P f6584f38965f4556b39abcc6c0e3eb5f: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:32:29.937053 31309 tablet_replica.cc:439] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } } }
I20260809 03:32:29.937160 31309 tablet_replica.cc:442] T 0371ce715c794dcb99e357c5b3f2c19d P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:29.937439 31336 mvcc.cc:205] Tried to move back new op lower bound from 7316465049316012032 to 7316465049271443456. Current Snapshot: MvccSnapshot[applied={T|T < 7316465049316012032 or (T in {7316465049316012032})}]
I20260809 03:32:29.939864 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.937340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35438 (local address 127.27.40.130:35553)
0809 03:32:29.937384 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.937386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.937428 (+ 42us) server_negotiation.cc:407] Connection header received
0809 03:32:29.937495 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.937496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.937506 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.937608 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:29.937762 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.938320 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.939109 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.939338 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.939414 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.939606 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:29.939644 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:29.939707 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.939770 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.939789 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 03:32:29.946691 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.943239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:47751 (local address 127.27.40.130:35553)
0809 03:32:29.943281 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.943283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.943453 (+ 170us) server_negotiation.cc:407] Connection header received
0809 03:32:29.943532 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.943533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.943542 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.943626 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:29.943773 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.944488 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.945825 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.946085 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.946170 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.946415 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:29.946445 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:29.946521 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.946588 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.946606 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 03:32:29.946980 31348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.943143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:35553 (local address 127.27.40.130:47751)
0809 03:32:29.943376 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:29.943387 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:29.943488 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:29.943628 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:29.943629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:29.943630 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:32:29.943708 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.943709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.944601 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.944602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.945727 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.945731 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.946170 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.946170 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.946282 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.946299 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:32:29.946524 (+ 225us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:32:29.946526 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 03:32:29.946584 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 03:32:29.946621 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":159,"threads_started":1}
I20260809 03:32:29.949834 31187 raft_consensus.cc:1261] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 FOLLOWER]: Refusing update from remote peer f6584f38965f4556b39abcc6c0e3eb5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:29.949872 30747 raft_consensus.cc:1261] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Refusing update from remote peer f6584f38965f4556b39abcc6c0e3eb5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:29.950120 31308 consensus_queue.cc:1035] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:29.950273 31312 consensus_queue.cc:1035] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:29.951738 31335 mvcc.cc:205] Tried to move back new op lower bound from 7316465049393049600 to 7316465048772313088. Current Snapshot: MvccSnapshot[applied={T|T < 7316465049393049600}]
I20260809 03:32:29.952067 31337 mvcc.cc:205] Tried to move back new op lower bound from 7316465049393049600 to 7316465048772313088. Current Snapshot: MvccSnapshot[applied={T|T < 7316465049393049600}]
I20260809 03:32:29.966571 30477 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:29.966692 30477 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:29.967511 30477 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3: Sending DeleteTablet for 3 replicas of tablet c6716a852ff847ccb2f2c3a3ff937458
I20260809 03:32:29.967813 30698 tablet_service.cc:1514] Processing DeleteTablet for tablet c6716a852ff847ccb2f2c3a3ff937458 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:29 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:35366
I20260809 03:32:29.967856 31160 tablet_service.cc:1514] Processing DeleteTablet for tablet c6716a852ff847ccb2f2c3a3ff937458 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:29 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:54182
I20260809 03:32:29.968114 30477 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 03:32:29.968164 30932 tablet_service.cc:1514] Processing DeleteTablet for tablet c6716a852ff847ccb2f2c3a3ff937458 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:29 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:35408
I20260809 03:32:29.968217 31355 tablet_replica.cc:330] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: stopping tablet replica
W20260809 03:32:29.968267 30477 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:29.968343 31355 raft_consensus.cc:2229] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:29.968421 31355 raft_consensus.cc:2258] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:29.968503 31356 tablet_replica.cc:330] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f: stopping tablet replica
I20260809 03:32:29.968604 31356 raft_consensus.cc:2229] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:32:29.968701 31356 raft_consensus.cc:2258] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:29.969111 31355 ts_tablet_manager.cc:1905] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:29.969249 31356 ts_tablet_manager.cc:1905] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:29.969882 30932 tablet_service.cc:1467] Processing CreateTablet for tablet 8ee32d7f201b43cfaa7514ee14b08d82 (DEFAULT_TABLE table=test-table [id=88b09df42eae4daf961403a8f88761a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:29.969882 31160 tablet_service.cc:1467] Processing CreateTablet for tablet 8ee32d7f201b43cfaa7514ee14b08d82 (DEFAULT_TABLE table=test-table [id=88b09df42eae4daf961403a8f88761a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:29.969884 30698 tablet_service.cc:1467] Processing CreateTablet for tablet 8ee32d7f201b43cfaa7514ee14b08d82 (DEFAULT_TABLE table=test-table [id=88b09df42eae4daf961403a8f88761a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:29.969996 30932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ee32d7f201b43cfaa7514ee14b08d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:29.969997 31160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ee32d7f201b43cfaa7514ee14b08d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:29.970005 30698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ee32d7f201b43cfaa7514ee14b08d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:29.971382 31298 tablet_bootstrap.cc:492] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49: Bootstrap starting.
I20260809 03:32:29.972306 31298 tablet_bootstrap.cc:654] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:29.972786 31299 tablet_bootstrap.cc:492] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Bootstrap starting.
I20260809 03:32:29.973172 31355 ts_tablet_manager.cc:1918] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:32:29.973242 31355 log.cc:1196] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/c6716a852ff847ccb2f2c3a3ff937458
I20260809 03:32:29.973304 31299 tablet_bootstrap.cc:654] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:29.973433 31356 ts_tablet_manager.cc:1918] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:32:29.973512 31356 log.cc:1196] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/c6716a852ff847ccb2f2c3a3ff937458
I20260809 03:32:29.973578 31355 ts_tablet_manager.cc:1939] T c6716a852ff847ccb2f2c3a3ff937458 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Deleting consensus metadata
I20260809 03:32:29.973807 31356 ts_tablet_manager.cc:1939] T c6716a852ff847ccb2f2c3a3ff937458 P f6584f38965f4556b39abcc6c0e3eb5f: Deleting consensus metadata
I20260809 03:32:29.973930 31296 tablet_bootstrap.cc:492] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f: Bootstrap starting.
I20260809 03:32:29.974248 30449 catalog_manager.cc:4841] TS f6584f38965f4556b39abcc6c0e3eb5f (127.27.40.130:35553): tablet c6716a852ff847ccb2f2c3a3ff937458 (table test-table [id=9d2789f3541d436798d56c63a93654e6]) successfully deleted
I20260809 03:32:29.974359 30454 catalog_manager.cc:4841] TS 9cb53b4ad9744b81a4ab8ec6cb2f8cca (127.27.40.131:45883): tablet c6716a852ff847ccb2f2c3a3ff937458 (table test-table [id=9d2789f3541d436798d56c63a93654e6]) successfully deleted
I20260809 03:32:29.974670 31296 tablet_bootstrap.cc:654] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:29.975608 31296 tablet_bootstrap.cc:492] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f: No bootstrap required, opened a new log
I20260809 03:32:29.975659 31296 ts_tablet_manager.cc:1397] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:32:29.975767 31298 tablet_bootstrap.cc:492] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49: No bootstrap required, opened a new log
I20260809 03:32:29.975764 31299 tablet_bootstrap.cc:492] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: No bootstrap required, opened a new log
I20260809 03:32:29.975809 31298 ts_tablet_manager.cc:1397] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 03:32:29.975829 31299 ts_tablet_manager.cc:1397] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 03:32:29.975800 31296 raft_consensus.cc:348] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.975998 31299 raft_consensus.cc:348] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.976068 31299 raft_consensus.cc:374] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:29.976070 31296 raft_consensus.cc:374] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:29.976091 31299 raft_consensus.cc:729] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb53b4ad9744b81a4ab8ec6cb2f8cca, State: Initialized, Role: FOLLOWER
I20260809 03:32:29.976101 31296 raft_consensus.cc:729] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6584f38965f4556b39abcc6c0e3eb5f, State: Initialized, Role: FOLLOWER
I20260809 03:32:29.976075 31298 raft_consensus.cc:348] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.976140 31299 consensus_queue.cc:260] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.976176 31298 raft_consensus.cc:374] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:29.976202 31298 raft_consensus.cc:729] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5390d00dd0d74888809954b8045b2d49, State: Initialized, Role: FOLLOWER
I20260809 03:32:29.976176 31296 consensus_queue.cc:260] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.976248 31298 consensus_queue.cc:260] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.976272 31296 ts_tablet_manager.cc:1428] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 03:32:29.976343 31298 ts_tablet_manager.cc:1428] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:29.976234 31299 ts_tablet_manager.cc:1428] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:29.977695 31354 tablet_replica.cc:330] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49: stopping tablet replica
I20260809 03:32:29.977847 31354 raft_consensus.cc:2229] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:29.977947 31354 raft_consensus.cc:2258] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:29.978400 31354 ts_tablet_manager.cc:1905] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:29.978570 31309 raft_consensus.cc:482] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:29.978629 31309 raft_consensus.cc:504] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.978926 31309 leader_election.cc:288] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5390d00dd0d74888809954b8045b2d49 (127.27.40.129:39783), f6584f38965f4556b39abcc6c0e3eb5f (127.27.40.130:35553)
I20260809 03:32:29.980635 31354 ts_tablet_manager.cc:1918] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:32:29.980743 31354 log.cc:1196] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/wal/wals/c6716a852ff847ccb2f2c3a3ff937458
I20260809 03:32:29.981078 31354 ts_tablet_manager.cc:1939] T c6716a852ff847ccb2f2c3a3ff937458 P 5390d00dd0d74888809954b8045b2d49: Deleting consensus metadata
I20260809 03:32:29.982475 30454 catalog_manager.cc:4841] TS 5390d00dd0d74888809954b8045b2d49 (127.27.40.129:39783): tablet c6716a852ff847ccb2f2c3a3ff937458 (table test-table [id=9d2789f3541d436798d56c63a93654e6]) successfully deleted
I20260809 03:32:29.983307 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.979762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:59257 (local address 127.27.40.130:35553)
0809 03:32:29.979813 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.979815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.979821 (+ 6us) server_negotiation.cc:407] Connection header received
0809 03:32:29.979842 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.979843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.979851 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.979906 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:29.980471 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.981211 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.982416 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.982864 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.982953 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.982966 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.983203 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.983241 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 03:32:29.983347 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.979511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:35553 (local address 127.27.40.131:59257)
0809 03:32:29.979664 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:29.979674 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:29.979724 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:29.979929 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:29.979930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:29.979932 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:29.980368 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.980372 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.981328 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.981328 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.982308 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.982311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.982978 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.982979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.983118 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.983122 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:29.983195 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 03:32:29.983239 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":46,"threads_started":1}
I20260809 03:32:29.983444 31096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.979151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:39783 (local address 127.27.40.131:50815)
0809 03:32:29.979246 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:29.979257 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:29.979299 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:29.979600 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:29.979601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:29.979603 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:29.979674 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.979676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.980843 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.980844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.982175 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:29.982178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.983201 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.983202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:29.983285 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.983289 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:29.983337 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 03:32:29.983375 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 03:32:29.983768 31292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.979372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:50815 (local address 127.27.40.129:39783)
0809 03:32:29.979439 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.979441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.979451 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:32:29.979476 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.979476 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.979486 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.979547 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:29.979951 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.980743 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.982532 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.983078 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.983157 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.983166 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.983640 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.983688 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 03:32:29.983692 30961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ee32d7f201b43cfaa7514ee14b08d82" candidate_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" is_pre_election: true
I20260809 03:32:29.983862 30961 raft_consensus.cc:2454] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb53b4ad9744b81a4ab8ec6cb2f8cca in term 0.
I20260809 03:32:29.983943 30747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ee32d7f201b43cfaa7514ee14b08d82" candidate_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5390d00dd0d74888809954b8045b2d49" is_pre_election: true
I20260809 03:32:29.984031 30747 raft_consensus.cc:2454] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb53b4ad9744b81a4ab8ec6cb2f8cca in term 0.
I20260809 03:32:29.984110 31085 leader_election.cc:302] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb53b4ad9744b81a4ab8ec6cb2f8cca, f6584f38965f4556b39abcc6c0e3eb5f; no voters:
I20260809 03:32:29.984227 31309 raft_consensus.cc:2781] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:29.984263 31309 raft_consensus.cc:482] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:29.984282 31309 raft_consensus.cc:3037] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:29.985011 31309 raft_consensus.cc:504] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.985157 31309 leader_election.cc:288] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [CANDIDATE]: Term 1 election: Requested vote from peers 5390d00dd0d74888809954b8045b2d49 (127.27.40.129:39783), f6584f38965f4556b39abcc6c0e3eb5f (127.27.40.130:35553)
I20260809 03:32:29.985306 30747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ee32d7f201b43cfaa7514ee14b08d82" candidate_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5390d00dd0d74888809954b8045b2d49"
I20260809 03:32:29.985322 30961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ee32d7f201b43cfaa7514ee14b08d82" candidate_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6584f38965f4556b39abcc6c0e3eb5f"
I20260809 03:32:29.985387 30747 raft_consensus.cc:3037] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:29.985390 30961 raft_consensus.cc:3037] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:29.986096 30961 raft_consensus.cc:2454] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb53b4ad9744b81a4ab8ec6cb2f8cca in term 1.
I20260809 03:32:29.986181 30747 raft_consensus.cc:2454] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb53b4ad9744b81a4ab8ec6cb2f8cca in term 1.
I20260809 03:32:29.986260 31085 leader_election.cc:302] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb53b4ad9744b81a4ab8ec6cb2f8cca, f6584f38965f4556b39abcc6c0e3eb5f; no voters:
I20260809 03:32:29.986340 31309 raft_consensus.cc:2781] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:29.986481 31309 raft_consensus.cc:686] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 LEADER]: Becoming Leader. State: Replica: 9cb53b4ad9744b81a4ab8ec6cb2f8cca, State: Running, Role: LEADER
I20260809 03:32:29.986531 31309 consensus_queue.cc:237] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:29.987339 30475 catalog_manager.cc:5495] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cb53b4ad9744b81a4ab8ec6cb2f8cca (127.27.40.131). New cstate: current_term: 1 leader_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } health_report { overall_health: HEALTHY } } }
I20260809 03:32:29.999248 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:29.996806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54198 (local address 127.27.40.131:45883)
0809 03:32:29.996844 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:29.996846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:29.996858 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:32:29.996937 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:29.996938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:29.996954 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:29.997015 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:29.997144 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.997677 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.998427 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:29.998636 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:29.998712 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:29.998883 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:29.998998 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:29.999082 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:29.999143 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 03:32:29.999165 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 03:32:30.004606 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:30.001971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:52415 (local address 127.27.40.130:35553)
0809 03:32:30.002019 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:30.002021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:30.002098 (+ 77us) server_negotiation.cc:407] Connection header received
0809 03:32:30.002175 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:30.002176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:30.002184 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:30.002229 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:30.002376 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:30.002811 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:30.003847 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:30.004089 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:30.004152 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:30.004375 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:30.004405 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:30.004446 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:30.004527 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 03:32:30.004543 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 03:32:30.004652 31276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:30.001905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:35553 (local address 127.27.40.131:52415)
0809 03:32:30.002047 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:30.002059 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:30.002112 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:30.002251 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:30.002251 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:30.002253 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:32:30.002319 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:30.002320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:30.002884 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:30.002885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:30.003753 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:30.003757 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:30.004179 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:30.004181 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:30.004271 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:30.004289 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:32:30.004469 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:32:30.004471 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 03:32:30.004525 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 03:32:30.004574 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260809 03:32:30.006834 30961 raft_consensus.cc:1261] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 FOLLOWER]: Refusing update from remote peer 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:30.006846 30747 raft_consensus.cc:1261] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Refusing update from remote peer 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:30.007153 31364 consensus_queue.cc:1035] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:30.007300 31309 consensus_queue.cc:1035] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:30.017395 31348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:30.013731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:45883 (local address 127.27.40.130:53409)
0809 03:32:30.013802 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:30.013814 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:30.013884 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:30.014156 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:30.014158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:30.014159 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:32:30.014237 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:30.014240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:30.015360 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:30.015361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:30.016376 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:30.016379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:30.016916 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:30.016917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:30.016992 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:30.016999 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:32:30.017217 (+ 218us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:32:30.017219 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 03:32:30.017266 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 03:32:30.017318 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 03:32:30.017406 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:30.013830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:53409 (local address 127.27.40.131:45883)
0809 03:32:30.013866 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:30.013868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:30.013877 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:30.013992 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:30.013993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:30.014006 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:30.014074 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:30.014340 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:30.015223 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:30.016467 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:30.016836 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:30.016902 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:30.017079 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:30.017111 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:30.017213 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:30.017288 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 03:32:30.017310 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 03:32:30.018198 31336 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 03:32:31.541298 30477 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:31.541419 30477 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:31.542481 30477 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3: Sending DeleteTablet for 3 replicas of tablet 8ee32d7f201b43cfaa7514ee14b08d82
I20260809 03:32:31.542821 30931 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ee32d7f201b43cfaa7514ee14b08d82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:31 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:35408
I20260809 03:32:31.542877 31160 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ee32d7f201b43cfaa7514ee14b08d82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:31 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:54182
I20260809 03:32:31.543040 30698 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ee32d7f201b43cfaa7514ee14b08d82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:31 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:35366
I20260809 03:32:31.544795 31410 tablet_replica.cc:330] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f: stopping tablet replica
I20260809 03:32:31.544927 31411 tablet_replica.cc:330] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: stopping tablet replica
I20260809 03:32:31.545073 31411 raft_consensus.cc:2229] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:32:31.545222 31411 raft_consensus.cc:2258] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:31.545812 31411 ts_tablet_manager.cc:1905] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:31.547345 31410 raft_consensus.cc:2229] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:31.547426 31410 raft_consensus.cc:2258] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:31.547590 31411 ts_tablet_manager.cc:1918] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 03:32:31.547664 31411 log.cc:1196] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/8ee32d7f201b43cfaa7514ee14b08d82
I20260809 03:32:31.547750 31410 ts_tablet_manager.cc:1905] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:31.547981 31411 ts_tablet_manager.cc:1939] T 8ee32d7f201b43cfaa7514ee14b08d82 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Deleting consensus metadata
I20260809 03:32:31.548334 30454 catalog_manager.cc:4841] TS 9cb53b4ad9744b81a4ab8ec6cb2f8cca (127.27.40.131:45883): tablet 8ee32d7f201b43cfaa7514ee14b08d82 (table test-table [id=88b09df42eae4daf961403a8f88761a3]) successfully deleted
I20260809 03:32:31.549681 31413 tablet_replica.cc:330] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49: stopping tablet replica
I20260809 03:32:31.549774 31413 raft_consensus.cc:2229] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:31.549837 31413 raft_consensus.cc:2258] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:31.549932 31410 ts_tablet_manager.cc:1918] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 03:32:31.550041 31410 log.cc:1196] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/8ee32d7f201b43cfaa7514ee14b08d82
I20260809 03:32:31.550118 31413 ts_tablet_manager.cc:1905] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:31.550313 31410 ts_tablet_manager.cc:1939] T 8ee32d7f201b43cfaa7514ee14b08d82 P f6584f38965f4556b39abcc6c0e3eb5f: Deleting consensus metadata
I20260809 03:32:31.550668 30449 catalog_manager.cc:4841] TS f6584f38965f4556b39abcc6c0e3eb5f (127.27.40.130:35553): tablet 8ee32d7f201b43cfaa7514ee14b08d82 (table test-table [id=88b09df42eae4daf961403a8f88761a3]) successfully deleted
I20260809 03:32:31.552147 31413 ts_tablet_manager.cc:1918] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 03:32:31.552219 31413 log.cc:1196] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/wal/wals/8ee32d7f201b43cfaa7514ee14b08d82
I20260809 03:32:31.552456 31413 ts_tablet_manager.cc:1939] T 8ee32d7f201b43cfaa7514ee14b08d82 P 5390d00dd0d74888809954b8045b2d49: Deleting consensus metadata
I20260809 03:32:31.552786 30454 catalog_manager.cc:4841] TS 5390d00dd0d74888809954b8045b2d49 (127.27.40.129:39783): tablet 8ee32d7f201b43cfaa7514ee14b08d82 (table test-table [id=88b09df42eae4daf961403a8f88761a3]) successfully deleted
Aug 09 03:32:31 dist-test-slave-gkw7 krb5kdc[30403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246351, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 03:32:31 dist-test-slave-gkw7 krb5kdc[30403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246351, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:31.574822 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:31.563755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52350 (local address 127.27.40.190:45839)
0809 03:32:31.563980 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:31.563982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:31.563991 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:31.564015 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:31.564016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:31.564026 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:31.564089 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:31.564710 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:31.565259 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:31.566176 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:31.566383 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:31.566439 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:31.572308 (+ 5869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:31.572310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:31.572310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:31.572318 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:31.573788 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:31.574126 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:31.574127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:31.574127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:31.574189 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:31.574368 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:31.574368 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:31.574369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:31.574479 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:31.574553 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:31.574675 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 03:32:31.574747 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":164,"threads_started":1}
I20260809 03:32:31.576668 30477 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:32:31.576869 30477 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:31.578497 30698 tablet_service.cc:1467] Processing CreateTablet for tablet 5e61195f57c8481bb6a480d360595635 (DEFAULT_TABLE table=test-table [id=5cd2abb1dc334a09ad4e996fef8e94c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:31.578507 31160 tablet_service.cc:1467] Processing CreateTablet for tablet 5e61195f57c8481bb6a480d360595635 (DEFAULT_TABLE table=test-table [id=5cd2abb1dc334a09ad4e996fef8e94c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:31.578533 30931 tablet_service.cc:1467] Processing CreateTablet for tablet 5e61195f57c8481bb6a480d360595635 (DEFAULT_TABLE table=test-table [id=5cd2abb1dc334a09ad4e996fef8e94c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:31.578662 30698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e61195f57c8481bb6a480d360595635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:31.578671 31160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e61195f57c8481bb6a480d360595635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:31.578658 30931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e61195f57c8481bb6a480d360595635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:31.579941 31427 tablet_bootstrap.cc:492] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Bootstrap starting.
I20260809 03:32:31.580129 31428 tablet_bootstrap.cc:492] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49: Bootstrap starting.
I20260809 03:32:31.580288 31431 tablet_bootstrap.cc:492] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f: Bootstrap starting.
I20260809 03:32:31.580389 31427 tablet_bootstrap.cc:654] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:31.580700 31431 tablet_bootstrap.cc:654] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:31.580849 31428 tablet_bootstrap.cc:654] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:31.581405 31431 tablet_bootstrap.cc:492] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f: No bootstrap required, opened a new log
I20260809 03:32:31.581468 31431 ts_tablet_manager.cc:1397] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:31.581503 31428 tablet_bootstrap.cc:492] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49: No bootstrap required, opened a new log
I20260809 03:32:31.581566 31428 ts_tablet_manager.cc:1397] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:31.581662 31431 raft_consensus.cc:348] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.581749 31431 raft_consensus.cc:374] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:31.581718 31428 raft_consensus.cc:348] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.581771 31431 raft_consensus.cc:729] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6584f38965f4556b39abcc6c0e3eb5f, State: Initialized, Role: FOLLOWER
I20260809 03:32:31.581813 31428 raft_consensus.cc:374] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:31.581836 31428 raft_consensus.cc:729] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5390d00dd0d74888809954b8045b2d49, State: Initialized, Role: FOLLOWER
I20260809 03:32:31.581840 31431 consensus_queue.cc:260] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.581879 31428 consensus_queue.cc:260] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.581986 31431 ts_tablet_manager.cc:1428] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:31.582050 31428 ts_tablet_manager.cc:1428] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:31.583802 31427 tablet_bootstrap.cc:492] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: No bootstrap required, opened a new log
I20260809 03:32:31.583865 31427 ts_tablet_manager.cc:1397] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 03:32:31.583969 31427 raft_consensus.cc:348] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.584048 31427 raft_consensus.cc:374] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:31.584103 31427 raft_consensus.cc:729] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb53b4ad9744b81a4ab8ec6cb2f8cca, State: Initialized, Role: FOLLOWER
I20260809 03:32:31.584158 31427 consensus_queue.cc:260] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.584277 31427 ts_tablet_manager.cc:1428] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:31.659796 31393 raft_consensus.cc:482] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:31.659921 31393 raft_consensus.cc:504] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.660138 31393 leader_election.cc:288] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5390d00dd0d74888809954b8045b2d49 (127.27.40.129:39783), f6584f38965f4556b39abcc6c0e3eb5f (127.27.40.130:35553)
I20260809 03:32:31.660447 30747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e61195f57c8481bb6a480d360595635" candidate_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5390d00dd0d74888809954b8045b2d49" is_pre_election: true
I20260809 03:32:31.660440 30961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e61195f57c8481bb6a480d360595635" candidate_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" is_pre_election: true
I20260809 03:32:31.660547 30747 raft_consensus.cc:2454] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb53b4ad9744b81a4ab8ec6cb2f8cca in term 0.
I20260809 03:32:31.660545 30961 raft_consensus.cc:2454] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb53b4ad9744b81a4ab8ec6cb2f8cca in term 0.
I20260809 03:32:31.660730 31088 leader_election.cc:302] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5390d00dd0d74888809954b8045b2d49, 9cb53b4ad9744b81a4ab8ec6cb2f8cca; no voters:
I20260809 03:32:31.660851 31393 raft_consensus.cc:2781] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:31.660935 31393 raft_consensus.cc:482] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:31.660964 31393 raft_consensus.cc:3037] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:31.661844 31393 raft_consensus.cc:504] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.662022 31393 leader_election.cc:288] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [CANDIDATE]: Term 1 election: Requested vote from peers 5390d00dd0d74888809954b8045b2d49 (127.27.40.129:39783), f6584f38965f4556b39abcc6c0e3eb5f (127.27.40.130:35553)
I20260809 03:32:31.662161 30747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e61195f57c8481bb6a480d360595635" candidate_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5390d00dd0d74888809954b8045b2d49"
I20260809 03:32:31.662163 30961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e61195f57c8481bb6a480d360595635" candidate_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6584f38965f4556b39abcc6c0e3eb5f"
I20260809 03:32:31.662271 30961 raft_consensus.cc:3037] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:31.662276 30747 raft_consensus.cc:3037] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:31.662947 30747 raft_consensus.cc:2454] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb53b4ad9744b81a4ab8ec6cb2f8cca in term 1.
I20260809 03:32:31.663107 30961 raft_consensus.cc:2454] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb53b4ad9744b81a4ab8ec6cb2f8cca in term 1.
I20260809 03:32:31.663137 31088 leader_election.cc:302] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5390d00dd0d74888809954b8045b2d49, 9cb53b4ad9744b81a4ab8ec6cb2f8cca; no voters:
I20260809 03:32:31.663223 31393 raft_consensus.cc:2781] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:31.663269 31393 raft_consensus.cc:686] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 LEADER]: Becoming Leader. State: Replica: 9cb53b4ad9744b81a4ab8ec6cb2f8cca, State: Running, Role: LEADER
I20260809 03:32:31.663323 31393 consensus_queue.cc:237] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.663986 30477 catalog_manager.cc:5495] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cb53b4ad9744b81a4ab8ec6cb2f8cca (127.27.40.131). New cstate: current_term: 1 leader_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } health_report { overall_health: HEALTHY } } }
I20260809 03:32:31.676589 31441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:31.674233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54212 (local address 127.27.40.131:45883)
0809 03:32:31.674386 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:31.674388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:31.674395 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:32:31.674413 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:31.674414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:31.674423 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:31.674479 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:31.674622 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:31.675123 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:31.675861 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:31.676048 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:31.676120 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:31.676238 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:31.676274 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:31.676320 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:31.676478 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 03:32:31.676497 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260809 03:32:31.679056 30961 raft_consensus.cc:1261] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 FOLLOWER]: Refusing update from remote peer 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:31.679083 30747 raft_consensus.cc:1261] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Refusing update from remote peer 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:31.679297 31400 consensus_queue.cc:1035] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:31.679409 31393 consensus_queue.cc:1035] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:31.690101 30477 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:31.690261 30477 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:31.690884 30477 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3: Sending DeleteTablet for 3 replicas of tablet 5e61195f57c8481bb6a480d360595635
I20260809 03:32:31.691119 30931 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e61195f57c8481bb6a480d360595635 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:31 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:35408
I20260809 03:32:31.691174 30698 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e61195f57c8481bb6a480d360595635 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:31 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:35366
I20260809 03:32:31.691290 31410 tablet_replica.cc:330] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f: stopping tablet replica
I20260809 03:32:31.691291 31413 tablet_replica.cc:330] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49: stopping tablet replica
I20260809 03:32:31.691355 31413 raft_consensus.cc:2229] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:31.691363 31410 raft_consensus.cc:2229] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:31.691414 31410 raft_consensus.cc:2258] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:31.691412 31413 raft_consensus.cc:2258] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:31.691516 30477 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 03:32:31.691583 31160 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e61195f57c8481bb6a480d360595635 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:31 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:54182
I20260809 03:32:31.691674 31411 tablet_replica.cc:330] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: stopping tablet replica
W20260809 03:32:31.691684 30477 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:31.691740 31411 raft_consensus.cc:2229] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:32:31.691771 31413 ts_tablet_manager.cc:1905] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:31.691798 31411 raft_consensus.cc:2258] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:31.692180 31410 ts_tablet_manager.cc:1905] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:31.693496 31411 ts_tablet_manager.cc:1905] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:31.693634 31413 ts_tablet_manager.cc:1918] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:32:31.693693 31413 log.cc:1196] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-0/wal/wals/5e61195f57c8481bb6a480d360595635
I20260809 03:32:31.693719 31410 ts_tablet_manager.cc:1918] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:32:31.693770 31410 log.cc:1196] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/5e61195f57c8481bb6a480d360595635
I20260809 03:32:31.693959 31413 ts_tablet_manager.cc:1939] T 5e61195f57c8481bb6a480d360595635 P 5390d00dd0d74888809954b8045b2d49: Deleting consensus metadata
I20260809 03:32:31.693990 31410 ts_tablet_manager.cc:1939] T 5e61195f57c8481bb6a480d360595635 P f6584f38965f4556b39abcc6c0e3eb5f: Deleting consensus metadata
I20260809 03:32:31.694286 30454 catalog_manager.cc:4841] TS 5390d00dd0d74888809954b8045b2d49 (127.27.40.129:39783): tablet 5e61195f57c8481bb6a480d360595635 (table test-table [id=5cd2abb1dc334a09ad4e996fef8e94c4]) successfully deleted
I20260809 03:32:31.694490 30449 catalog_manager.cc:4841] TS f6584f38965f4556b39abcc6c0e3eb5f (127.27.40.130:35553): tablet 5e61195f57c8481bb6a480d360595635 (table test-table [id=5cd2abb1dc334a09ad4e996fef8e94c4]) successfully deleted
I20260809 03:32:31.694499 30698 tablet_service.cc:1467] Processing CreateTablet for tablet eb3ea95d31b1400cabb4942c277bbbd6 (DEFAULT_TABLE table=test-table [id=7e496a64a9f142ba857b611805232d6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:31.694558 30931 tablet_service.cc:1467] Processing CreateTablet for tablet eb3ea95d31b1400cabb4942c277bbbd6 (DEFAULT_TABLE table=test-table [id=7e496a64a9f142ba857b611805232d6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:31.694607 30698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb3ea95d31b1400cabb4942c277bbbd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:31.694661 30931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb3ea95d31b1400cabb4942c277bbbd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:31.695053 31411 ts_tablet_manager.cc:1918] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:32:31.695096 31411 log.cc:1196] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/5e61195f57c8481bb6a480d360595635
I20260809 03:32:31.695252 31411 ts_tablet_manager.cc:1939] T 5e61195f57c8481bb6a480d360595635 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Deleting consensus metadata
I20260809 03:32:31.695581 31160 tablet_service.cc:1467] Processing CreateTablet for tablet eb3ea95d31b1400cabb4942c277bbbd6 (DEFAULT_TABLE table=test-table [id=7e496a64a9f142ba857b611805232d6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:31.695691 31160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb3ea95d31b1400cabb4942c277bbbd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:31.695739 30454 catalog_manager.cc:4841] TS 9cb53b4ad9744b81a4ab8ec6cb2f8cca (127.27.40.131:45883): tablet 5e61195f57c8481bb6a480d360595635 (table test-table [id=5cd2abb1dc334a09ad4e996fef8e94c4]) successfully deleted
I20260809 03:32:31.695844 31428 tablet_bootstrap.cc:492] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49: Bootstrap starting.
I20260809 03:32:31.695910 31431 tablet_bootstrap.cc:492] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f: Bootstrap starting.
I20260809 03:32:31.696254 31431 tablet_bootstrap.cc:654] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:31.696460 31428 tablet_bootstrap.cc:654] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:31.696775 31427 tablet_bootstrap.cc:492] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Bootstrap starting.
I20260809 03:32:31.696889 31431 tablet_bootstrap.cc:492] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f: No bootstrap required, opened a new log
I20260809 03:32:31.696935 31431 ts_tablet_manager.cc:1397] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 03:32:31.697034 31431 raft_consensus.cc:348] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.697081 31431 raft_consensus.cc:374] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:31.697094 31431 raft_consensus.cc:729] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6584f38965f4556b39abcc6c0e3eb5f, State: Initialized, Role: FOLLOWER
I20260809 03:32:31.697088 31427 tablet_bootstrap.cc:654] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:31.697154 31431 consensus_queue.cc:260] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.697268 31431 ts_tablet_manager.cc:1428] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:31.697322 31428 tablet_bootstrap.cc:492] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49: No bootstrap required, opened a new log
I20260809 03:32:31.697363 31428 ts_tablet_manager.cc:1397] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:32:31.697522 31428 raft_consensus.cc:348] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.697628 31428 raft_consensus.cc:374] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:31.697659 31428 raft_consensus.cc:729] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5390d00dd0d74888809954b8045b2d49, State: Initialized, Role: FOLLOWER
I20260809 03:32:31.697700 31428 consensus_queue.cc:260] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.697801 31428 ts_tablet_manager.cc:1428] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:31.697903 31427 tablet_bootstrap.cc:492] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: No bootstrap required, opened a new log
I20260809 03:32:31.697961 31427 ts_tablet_manager.cc:1397] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:31.698074 31427 raft_consensus.cc:348] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.698144 31427 raft_consensus.cc:374] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:31.698163 31427 raft_consensus.cc:729] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb53b4ad9744b81a4ab8ec6cb2f8cca, State: Initialized, Role: FOLLOWER
I20260809 03:32:31.698216 31427 consensus_queue.cc:260] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.698326 31427 ts_tablet_manager.cc:1428] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:31.730619 31308 raft_consensus.cc:482] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:31.730764 31308 raft_consensus.cc:504] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.730933 31308 leader_election.cc:288] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5390d00dd0d74888809954b8045b2d49 (127.27.40.129:39783), 9cb53b4ad9744b81a4ab8ec6cb2f8cca (127.27.40.131:45883)
I20260809 03:32:31.731171 30747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb3ea95d31b1400cabb4942c277bbbd6" candidate_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5390d00dd0d74888809954b8045b2d49" is_pre_election: true
I20260809 03:32:31.731180 31187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb3ea95d31b1400cabb4942c277bbbd6" candidate_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" is_pre_election: true
I20260809 03:32:31.731288 31187 raft_consensus.cc:2454] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6584f38965f4556b39abcc6c0e3eb5f in term 0.
I20260809 03:32:31.731288 30747 raft_consensus.cc:2454] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6584f38965f4556b39abcc6c0e3eb5f in term 0.
I20260809 03:32:31.731473 30861 leader_election.cc:302] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb53b4ad9744b81a4ab8ec6cb2f8cca, f6584f38965f4556b39abcc6c0e3eb5f; no voters:
I20260809 03:32:31.731590 31308 raft_consensus.cc:2781] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:31.731680 31308 raft_consensus.cc:482] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:31.731714 31308 raft_consensus.cc:3037] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:31.732401 31308 raft_consensus.cc:504] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.732547 31308 leader_election.cc:288] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [CANDIDATE]: Term 1 election: Requested vote from peers 5390d00dd0d74888809954b8045b2d49 (127.27.40.129:39783), 9cb53b4ad9744b81a4ab8ec6cb2f8cca (127.27.40.131:45883)
I20260809 03:32:31.732756 30747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb3ea95d31b1400cabb4942c277bbbd6" candidate_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5390d00dd0d74888809954b8045b2d49"
I20260809 03:32:31.732769 31187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb3ea95d31b1400cabb4942c277bbbd6" candidate_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca"
I20260809 03:32:31.732843 31187 raft_consensus.cc:3037] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:31.732843 30747 raft_consensus.cc:3037] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:31.733724 30747 raft_consensus.cc:2454] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6584f38965f4556b39abcc6c0e3eb5f in term 1.
I20260809 03:32:31.733753 31187 raft_consensus.cc:2454] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6584f38965f4556b39abcc6c0e3eb5f in term 1.
I20260809 03:32:31.733912 30861 leader_election.cc:302] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb53b4ad9744b81a4ab8ec6cb2f8cca, f6584f38965f4556b39abcc6c0e3eb5f; no voters:
I20260809 03:32:31.734009 31308 raft_consensus.cc:2781] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:31.734077 31308 raft_consensus.cc:686] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 LEADER]: Becoming Leader. State: Replica: f6584f38965f4556b39abcc6c0e3eb5f, State: Running, Role: LEADER
I20260809 03:32:31.734134 31308 consensus_queue.cc:237] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } }
I20260809 03:32:31.734738 30477 catalog_manager.cc:5495] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f reported cstate change: term changed from 0 to 1, leader changed from <none> to f6584f38965f4556b39abcc6c0e3eb5f (127.27.40.130). New cstate: current_term: 1 leader_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6584f38965f4556b39abcc6c0e3eb5f" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 } health_report { overall_health: UNKNOWN } } }
I20260809 03:32:31.745926 31453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:31.743555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35446 (local address 127.27.40.130:35553)
0809 03:32:31.743700 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:31.743701 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:31.743707 (+ 6us) server_negotiation.cc:407] Connection header received
0809 03:32:31.743725 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:31.743726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:31.743735 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:31.743813 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:31.743955 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:31.744454 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:31.745190 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:31.745399 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:31.745483 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:31.745639 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:31.745678 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:31.745743 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:31.745805 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 03:32:31.745829 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":93,"threads_started":1}
I20260809 03:32:31.748312 31187 raft_consensus.cc:1261] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 FOLLOWER]: Refusing update from remote peer f6584f38965f4556b39abcc6c0e3eb5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:31.748337 30747 raft_consensus.cc:1261] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Refusing update from remote peer f6584f38965f4556b39abcc6c0e3eb5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:31.748554 31308 consensus_queue.cc:1035] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5390d00dd0d74888809954b8045b2d49" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:31.748703 31376 consensus_queue.cc:1035] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb53b4ad9744b81a4ab8ec6cb2f8cca" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 03:32:31.752967 31374 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 03:32:33.271572 30477 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:33.271685 30477 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:33.272713 30477 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d64bc39257074a988e422bb5dd2b28e3: Sending DeleteTablet for 3 replicas of tablet eb3ea95d31b1400cabb4942c277bbbd6
I20260809 03:32:33.273285 31160 tablet_service.cc:1514] Processing DeleteTablet for tablet eb3ea95d31b1400cabb4942c277bbbd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:33 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:54182
I20260809 03:32:33.273528 31482 tablet_replica.cc:330] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: stopping tablet replica
I20260809 03:32:33.273622 31482 raft_consensus.cc:2229] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:33.273663 31482 raft_consensus.cc:2258] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:33.273022 30698 tablet_service.cc:1514] Processing DeleteTablet for tablet eb3ea95d31b1400cabb4942c277bbbd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:33 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:35366
I20260809 03:32:33.273962 31482 ts_tablet_manager.cc:1905] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:33.274228 30931 tablet_service.cc:1514] Processing DeleteTablet for tablet eb3ea95d31b1400cabb4942c277bbbd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:33 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:35408
I20260809 03:32:33.274430 31483 tablet_replica.cc:330] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f: stopping tablet replica
I20260809 03:32:33.274499 31483 raft_consensus.cc:2229] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:32:33.274572 31483 raft_consensus.cc:2258] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:33.274837 31483 ts_tablet_manager.cc:1905] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:33.275491 31484 tablet_replica.cc:330] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49: stopping tablet replica
I20260809 03:32:33.275557 31484 raft_consensus.cc:2229] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:33.275591 31484 raft_consensus.cc:2258] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:33.275243 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 30591
I20260809 03:32:33.275846 31484 ts_tablet_manager.cc:1905] T eb3ea95d31b1400cabb4942c277bbbd6 P 5390d00dd0d74888809954b8045b2d49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:33.281569 31483 ts_tablet_manager.cc:1918] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 03:32:33.281656 31483 log.cc:1196] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/eb3ea95d31b1400cabb4942c277bbbd6
I20260809 03:32:33.282131 31483 ts_tablet_manager.cc:1939] T eb3ea95d31b1400cabb4942c277bbbd6 P f6584f38965f4556b39abcc6c0e3eb5f: Deleting consensus metadata
I20260809 03:32:33.282630 30449 catalog_manager.cc:4841] TS f6584f38965f4556b39abcc6c0e3eb5f (127.27.40.130:35553): tablet eb3ea95d31b1400cabb4942c277bbbd6 (table test-table [id=7e496a64a9f142ba857b611805232d6f]) successfully deleted
I20260809 03:32:33.285768 31482 ts_tablet_manager.cc:1918] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 03:32:33.285853 31482 log.cc:1196] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/eb3ea95d31b1400cabb4942c277bbbd6
I20260809 03:32:33.286110 31482 ts_tablet_manager.cc:1939] T eb3ea95d31b1400cabb4942c277bbbd6 P 9cb53b4ad9744b81a4ab8ec6cb2f8cca: Deleting consensus metadata
I20260809 03:32:33.286546 30454 catalog_manager.cc:4841] TS 9cb53b4ad9744b81a4ab8ec6cb2f8cca (127.27.40.131:45883): tablet eb3ea95d31b1400cabb4942c277bbbd6 (table test-table [id=7e496a64a9f142ba857b611805232d6f]) successfully deleted
W20260809 03:32:33.297328 30454 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.40.129:39783 (error 108)
I20260809 03:32:33.297713 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 30830
I20260809 03:32:33.300313 31486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:33.297835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:39783 (local address 127.0.0.1:35382)
0809 03:32:33.300188 (+ 2353us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:33.300252 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:39783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2318,"thread_start_us":74,"threads_started":1}
W20260809 03:32:33.300474 30454 catalog_manager.cc:4584] TS 5390d00dd0d74888809954b8045b2d49 (127.27.40.129:39783): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eb3ea95d31b1400cabb4942c277bbbd6: Network error: Client connection negotiation failed: client connection to 127.27.40.129:39783: connect: Connection refused (error 111)
I20260809 03:32:33.310711 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 31060
I20260809 03:32:33.324498 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 30428
2026-08-09T03:32:33Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5810 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:32:33 dist-test-slave-gkw7 krb5kdc[31492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:32:33 dist-test-slave-gkw7 krb5kdc[31492](info): set up 2 sockets
Aug 09 03:32:33 dist-test-slave-gkw7 krb5kdc[31492](info): commencing operation
krb5kdc: starting...
W20260809 03:32:34.575578 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:32:34 dist-test-slave-gkw7 krb5kdc[31492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:32:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:32:34.704130 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:34123
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:34123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:34.786214 31509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:34.786470 31509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:34.786507 31509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:34.787611 31509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:34.787690 31509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:34.787714 31509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:34.787753 31509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:34.787786 31509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:32:34.789259 31509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:34123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:34123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:34.789556 31509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:34.789803 31509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:34.795156 31516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:34.795131 31515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:34.795164 31509 server_base.cc:1034] running on GCE node
W20260809 03:32:34.795135 31520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:34.795625 31509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:34.795945 31509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:34.797117 31509 hybrid_clock.cc:648] HybridClock initialized: now 1786246354797100 us; error 30 us; skew 500 ppm
Aug 09 03:32:34 dist-test-slave-gkw7 krb5kdc[31492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246354, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:34.799723 31509 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:32:34.800107 31509 webserver.cc:460] Webserver started at http://127.27.40.190:42115/ using document root <none> and password file <none>
I20260809 03:32:34.800338 31509 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:34.800405 31509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:34.800557 31509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:34.801703 31509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "90c8781c38d049778eeb7906417cdf2c"
format_stamp: "Formatted at 2026-08-09 03:32:34 on dist-test-slave-gkw7"
server_key: "940f0e25fe7c7fb5b0dcaf074f5df7fc"
server_key_iv: "8b878232bf6bce072eaf1ecda3af9f5d"
server_key_version: "encryptionkey@0"
I20260809 03:32:34.802191 31509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "90c8781c38d049778eeb7906417cdf2c"
format_stamp: "Formatted at 2026-08-09 03:32:34 on dist-test-slave-gkw7"
server_key: "940f0e25fe7c7fb5b0dcaf074f5df7fc"
server_key_iv: "8b878232bf6bce072eaf1ecda3af9f5d"
server_key_version: "encryptionkey@0"
I20260809 03:32:34.803663 31509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 03:32:34.804479 31530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:34.804695 31509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:32:34.804801 31509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "90c8781c38d049778eeb7906417cdf2c"
format_stamp: "Formatted at 2026-08-09 03:32:34 on dist-test-slave-gkw7"
server_key: "940f0e25fe7c7fb5b0dcaf074f5df7fc"
server_key_iv: "8b878232bf6bce072eaf1ecda3af9f5d"
server_key_version: "encryptionkey@0"
I20260809 03:32:34.804873 31509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:34.815181 31509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:34.817000 31509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:34.817174 31509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:34.821401 31509 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:34123
I20260809 03:32:34.821434 31617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:34123 every 8 connection(s)
I20260809 03:32:34.821893 31509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:34.822769 31619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:34.825060 31619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c: Bootstrap starting.
I20260809 03:32:34.825750 31619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:34.826056 31619 log.cc:824] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:34.826772 31619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c: No bootstrap required, opened a new log
I20260809 03:32:34.828365 31619 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c8781c38d049778eeb7906417cdf2c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34123 } }
I20260809 03:32:34.828500 31619 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:34.828523 31619 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c8781c38d049778eeb7906417cdf2c, State: Initialized, Role: FOLLOWER
I20260809 03:32:34.828698 31619 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c8781c38d049778eeb7906417cdf2c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34123 } }
I20260809 03:32:34.828770 31619 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:34.828791 31619 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:34.828841 31619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:34.829298 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 31509
I20260809 03:32:34.829403 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:34.829363 31619 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c8781c38d049778eeb7906417cdf2c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34123 } }
I20260809 03:32:34.829497 31619 leader_election.cc:302] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90c8781c38d049778eeb7906417cdf2c; no voters:
I20260809 03:32:34.829653 27810 external_mini_cluster.cc:1428] Setting key be25240fd456559f9af6852d6577ddd6
I20260809 03:32:34.829706 31619 leader_election.cc:288] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:34.829730 31623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:34.829859 31623 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [term 1 LEADER]: Becoming Leader. State: Replica: 90c8781c38d049778eeb7906417cdf2c, State: Running, Role: LEADER
I20260809 03:32:34.829990 31619 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:34.829942 31623 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c8781c38d049778eeb7906417cdf2c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34123 } }
I20260809 03:32:34.830355 31624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90c8781c38d049778eeb7906417cdf2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c8781c38d049778eeb7906417cdf2c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34123 } } }
I20260809 03:32:34.830463 31624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:34.830343 31623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90c8781c38d049778eeb7906417cdf2c. Latest consensus state: current_term: 1 leader_uuid: "90c8781c38d049778eeb7906417cdf2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c8781c38d049778eeb7906417cdf2c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34123 } } }
I20260809 03:32:34.830626 31623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:34.830760 31629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:34.831566 31629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:34.834900 31629 catalog_manager.cc:1347] Generated new cluster ID: 2b652c7cbe4a4f26bf068f1ab8e8ac05
I20260809 03:32:34.834955 31629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 03:32:34 dist-test-slave-gkw7 krb5kdc[31492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:34.841701 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:34.830172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52232 (local address 127.27.40.190:34123)
0809 03:32:34.830426 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:34.830435 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:34.830691 (+ 256us) server_negotiation.cc:407] Connection header received
0809 03:32:34.830814 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:34.830822 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:34.830885 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:34.831023 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:34.832044 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:34.833413 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:34.834499 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:34.834999 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:34.835146 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:34.837974 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:34.837979 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:34.837979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:34.837991 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:34.840035 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:34.840419 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:34.840422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:34.840423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:34.840545 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:34.840745 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:34.840746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:34.840747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:34.841044 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:34.841212 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:34.841309 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 03:32:34.841449 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":83,"threads_started":1}
I20260809 03:32:34.855697 31629 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:34.856212 31629 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:34.862205 31629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90c8781c38d049778eeb7906417cdf2c: Generated new TSK 0
I20260809 03:32:34.862404 31629 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:32:34.900017 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34123
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:34.983142 31649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:34.983359 31649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:34.983382 31649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:34.984537 31649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:34.984579 31649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:34.984606 31649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:32:34.986105 31649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:34.986385 31649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:34.986626 31649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:34.991701 31660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:34.991712 31664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:34.991824 31662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:34.992259 31649 server_base.cc:1034] running on GCE node
I20260809 03:32:34.992589 31649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:34.992810 31649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:34.993945 31649 hybrid_clock.cc:648] HybridClock initialized: now 1786246354993935 us; error 33 us; skew 500 ppm
Aug 09 03:32:34 dist-test-slave-gkw7 krb5kdc[31492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246354, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:34.996471 31649 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:32:34.996958 31649 webserver.cc:460] Webserver started at http://127.27.40.129:34503/ using document root <none> and password file <none>
I20260809 03:32:34.997162 31649 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:34.997203 31649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:34.997296 31649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:34.998350 31649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "5a7b0593330a431f9cdf81cecb7bc84c"
format_stamp: "Formatted at 2026-08-09 03:32:34 on dist-test-slave-gkw7"
server_key: "1a155ea1c32d4a8b46b5bf19bedef58e"
server_key_iv: "33c79295bb0975ae9cacf0f0735dadb9"
server_key_version: "encryptionkey@0"
I20260809 03:32:34.998656 31649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "5a7b0593330a431f9cdf81cecb7bc84c"
format_stamp: "Formatted at 2026-08-09 03:32:34 on dist-test-slave-gkw7"
server_key: "1a155ea1c32d4a8b46b5bf19bedef58e"
server_key_iv: "33c79295bb0975ae9cacf0f0735dadb9"
server_key_version: "encryptionkey@0"
I20260809 03:32:34.999954 31649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 03:32:35.000705 31678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:35.000901 31649 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:32:35.000994 31649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "5a7b0593330a431f9cdf81cecb7bc84c"
format_stamp: "Formatted at 2026-08-09 03:32:34 on dist-test-slave-gkw7"
server_key: "1a155ea1c32d4a8b46b5bf19bedef58e"
server_key_iv: "33c79295bb0975ae9cacf0f0735dadb9"
server_key_version: "encryptionkey@0"
I20260809 03:32:35.001068 31649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:35.015666 31649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:35.018045 31649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:35.018201 31649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:35.018451 31649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:35.018779 31649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:35.018836 31649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:35.018872 31649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:35.018899 31649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:35.024824 31649 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:45133
I20260809 03:32:35.024883 31856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:45133 every 8 connection(s)
I20260809 03:32:35.025226 31649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:32:35.025444 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 31649
I20260809 03:32:35.025568 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:32:35.025794 27810 external_mini_cluster.cc:1428] Setting key 303f748be90760a16c9f953394f4dfa4
Aug 09 03:32:35 dist-test-slave-gkw7 krb5kdc[31492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246354, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
I20260809 03:32:35.036262 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:35.026087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:48021 (local address 127.27.40.190:34123)
0809 03:32:35.026146 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:35.026150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:35.026401 (+ 251us) server_negotiation.cc:407] Connection header received
0809 03:32:35.026937 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:35.026939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:35.026951 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:35.027018 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:35.027938 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:35.028590 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:35.030176 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:35.030375 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:35.030435 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:35.033148 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:35.033150 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:35.033150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:35.033157 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:35.034622 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:35.035141 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:35.035142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:35.035142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:35.035204 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:35.035507 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:35.035508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:35.035508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:35.035624 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:35.035689 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:35.036110 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 03:32:35.036192 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 03:32:35.036410 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:35.026083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34123 (local address 127.27.40.129:48021)
0809 03:32:35.026255 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:35.026298 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:35.026834 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:35.027080 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:35.027083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:35.027646 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:35.027842 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:35.027846 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:35.028687 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:35.028688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:35.030076 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:35.030079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:35.030467 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:35.030469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:35.030646 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:35.031239 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:35.031251 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:35.033018 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:35.034711 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:35.034719 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:35.034722 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:35.035038 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:35.035285 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:35.035286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:35.035287 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:35.035438 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:35.035752 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:35.035760 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:35.035935 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 03:32:35.036059 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 03:32:35.036201 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260809 03:32:35.036996 31857 heartbeater.cc:347] Connected to a master server at 127.27.40.190:34123
I20260809 03:32:35.037101 31857 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:35.037257 31857 heartbeater.cc:511] Master 127.27.40.190:34123 requested a full tablet report, sending...
I20260809 03:32:35.037870 31558 ts_manager.cc:194] Registered new tserver with Master: 5a7b0593330a431f9cdf81cecb7bc84c (127.27.40.129:45133)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
I20260809 03:32:35.038765 31558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:48021
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:32:35.055787 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34123
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:35.139041 31864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:35.139243 31864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:35.139266 31864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:35.140336 31864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:35.140372 31864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:35.140398 31864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:32:35.141889 31864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:35.142062 31864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:35.142239 31864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:35.147454 31871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:35.147423 31872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:35.147428 31877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:35.147598 31864 server_base.cc:1034] running on GCE node
I20260809 03:32:35.147871 31864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:35.148108 31864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:35.149286 31864 hybrid_clock.cc:648] HybridClock initialized: now 1786246355149274 us; error 28 us; skew 500 ppm
Aug 09 03:32:35 dist-test-slave-gkw7 krb5kdc[31492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246355, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:35.151871 31864 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:32:35.152369 31864 webserver.cc:460] Webserver started at http://127.27.40.130:46515/ using document root <none> and password file <none>
I20260809 03:32:35.152602 31864 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:35.152675 31864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:35.152810 31864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:35.153927 31864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "eacfe5567fa44bcf9a78c756db23110a"
format_stamp: "Formatted at 2026-08-09 03:32:35 on dist-test-slave-gkw7"
server_key: "d9269898f4bd0d44c72173817c4d841c"
server_key_iv: "da489df767b80476ab7f230b61c7bc06"
server_key_version: "encryptionkey@0"
I20260809 03:32:35.154296 31864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "eacfe5567fa44bcf9a78c756db23110a"
format_stamp: "Formatted at 2026-08-09 03:32:35 on dist-test-slave-gkw7"
server_key: "d9269898f4bd0d44c72173817c4d841c"
server_key_iv: "da489df767b80476ab7f230b61c7bc06"
server_key_version: "encryptionkey@0"
I20260809 03:32:35.155746 31864 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:32:35.156579 31884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:35.156788 31864 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:35.156899 31864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "eacfe5567fa44bcf9a78c756db23110a"
format_stamp: "Formatted at 2026-08-09 03:32:35 on dist-test-slave-gkw7"
server_key: "d9269898f4bd0d44c72173817c4d841c"
server_key_iv: "da489df767b80476ab7f230b61c7bc06"
server_key_version: "encryptionkey@0"
I20260809 03:32:35.156973 31864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:35.176594 31864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:35.178674 31864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:35.178851 31864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:35.179090 31864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:35.179432 31864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:35.179490 31864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:35.179533 31864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:35.179561 31864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:35.185348 31864 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:40525
I20260809 03:32:35.185398 32058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:40525 every 8 connection(s)
I20260809 03:32:35.185725 31864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:32:35.191294 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 31864
I20260809 03:32:35.191407 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:32:35.191602 27810 external_mini_cluster.cc:1428] Setting key f30cb2b2de97276eed0b59ab5667ae36
Aug 09 03:32:35 dist-test-slave-gkw7 krb5kdc[31492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246355, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:35.195222 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:35.186453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:35149 (local address 127.27.40.190:34123)
0809 03:32:35.186520 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:35.186522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:35.186740 (+ 218us) server_negotiation.cc:407] Connection header received
0809 03:32:35.187216 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:35.187217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:35.187241 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:35.187295 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:35.188351 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:35.188841 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:35.189863 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:35.190058 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:35.190113 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:35.192354 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:35.192356 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:35.192357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:35.192366 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:35.193866 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:35.194303 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:35.194304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:35.194305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:35.194371 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:35.194600 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:35.194601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:35.194601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:35.194748 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:35.194819 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:35.195045 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 03:32:35.195149 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 03:32:35.195310 32062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:35.186436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34123 (local address 127.27.40.130:35149)
0809 03:32:35.186616 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:35.186658 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:35.187121 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:35.187376 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:35.187379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:35.187952 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:35.188275 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:35.188280 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:35.188917 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:35.188917 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:35.189792 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:35.189794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:35.190165 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:35.190166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:35.190276 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:35.190797 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:35.190805 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:35.192252 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:35.193974 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:35.193982 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:35.193984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:35.194225 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:35.194437 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:35.194437 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:35.194438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:35.194536 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:35.194811 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:35.194815 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:35.194921 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 03:32:35.195001 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 03:32:35.195116 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
I20260809 03:32:35.195847 32059 heartbeater.cc:347] Connected to a master server at 127.27.40.190:34123
I20260809 03:32:35.195953 32059 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:35.196141 32059 heartbeater.cc:511] Master 127.27.40.190:34123 requested a full tablet report, sending...
I20260809 03:32:35.196496 31558 ts_manager.cc:194] Registered new tserver with Master: eacfe5567fa44bcf9a78c756db23110a (127.27.40.130:40525)
I20260809 03:32:35.196892 31558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:35149
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:32:35.220741 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34123
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:35.305816 32068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:35.306005 32068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:35.306028 32068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:35.307109 32068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:35.307142 32068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:35.307168 32068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:32:35.308529 32068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:35.308715 32068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:35.308884 32068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:35.313786 32078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:35.313786 32074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:35.313827 32075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:35.314097 32068 server_base.cc:1034] running on GCE node
I20260809 03:32:35.314316 32068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:35.314558 32068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:35.315697 32068 hybrid_clock.cc:648] HybridClock initialized: now 1786246355315682 us; error 43 us; skew 500 ppm
Aug 09 03:32:35 dist-test-slave-gkw7 krb5kdc[31492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246355, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:35.318261 32068 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:32:35.318672 32068 webserver.cc:460] Webserver started at http://127.27.40.131:44399/ using document root <none> and password file <none>
I20260809 03:32:35.318882 32068 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:35.318923 32068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:35.319010 32068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:35.319959 32068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "278f4ff7f54d458ab0a4a5d3fce58901"
format_stamp: "Formatted at 2026-08-09 03:32:35 on dist-test-slave-gkw7"
server_key: "6e9488e2f78c9e57e4e952e9d49773f6"
server_key_iv: "bde7ab0cb5a303f428cef954d841034d"
server_key_version: "encryptionkey@0"
I20260809 03:32:35.320247 32068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "278f4ff7f54d458ab0a4a5d3fce58901"
format_stamp: "Formatted at 2026-08-09 03:32:35 on dist-test-slave-gkw7"
server_key: "6e9488e2f78c9e57e4e952e9d49773f6"
server_key_iv: "bde7ab0cb5a303f428cef954d841034d"
server_key_version: "encryptionkey@0"
I20260809 03:32:35.321501 32068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 03:32:35.322454 32089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:35.322659 32068 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:35.322764 32068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "278f4ff7f54d458ab0a4a5d3fce58901"
format_stamp: "Formatted at 2026-08-09 03:32:35 on dist-test-slave-gkw7"
server_key: "6e9488e2f78c9e57e4e952e9d49773f6"
server_key_iv: "bde7ab0cb5a303f428cef954d841034d"
server_key_version: "encryptionkey@0"
I20260809 03:32:35.322847 32068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:35.342199 32068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:35.345502 32068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:35.345700 32068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:35.345959 32068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:35.346313 32068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:35.346349 32068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:35.346375 32068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:35.346395 32068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:35.352924 32068 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:36199
I20260809 03:32:35.353005 32249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:36199 every 8 connection(s)
I20260809 03:32:35.353288 32068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:32:35.356246 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 32068
I20260809 03:32:35.356321 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:32:35.356513 27810 external_mini_cluster.cc:1428] Setting key 44bea2c8dda6b47dcec378c3febd59dc
Aug 09 03:32:35 dist-test-slave-gkw7 krb5kdc[31492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246355, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:35.363188 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:35.354022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:36153 (local address 127.27.40.190:34123)
0809 03:32:35.354084 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:35.354086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:35.354304 (+ 218us) server_negotiation.cc:407] Connection header received
0809 03:32:35.354835 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:35.354836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:35.354850 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:35.354901 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:35.355853 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:35.356453 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:35.357863 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:35.358082 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:35.358163 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:35.360326 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:35.360328 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:35.360328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:35.360337 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:35.361697 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:35.362136 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:35.362138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:35.362138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:35.362208 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:35.362498 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:35.362499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:35.362499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:35.362633 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:35.362730 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:35.363013 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 03:32:35.363094 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 03:32:35.363427 32253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:35.354001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34123 (local address 127.27.40.131:36153)
0809 03:32:35.354183 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:35.354217 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:35.354720 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:35.354975 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:35.354979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:35.355553 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:35.355751 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:35.355755 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:35.356550 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:35.356551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:35.357777 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:35.357780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:35.358171 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:35.358171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:35.358314 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:35.358749 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:35.358758 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:35.360228 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:35.361779 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:35.361799 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:35.361801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:35.362031 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:35.362305 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:35.362306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:35.362306 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:35.362427 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:35.362723 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:35.362731 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:35.362871 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 03:32:35.362977 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 03:32:35.363150 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":96,"threads_started":1}
I20260809 03:32:35.363888 32251 heartbeater.cc:347] Connected to a master server at 127.27.40.190:34123
I20260809 03:32:35.363979 32251 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:35.364130 32251 heartbeater.cc:511] Master 127.27.40.190:34123 requested a full tablet report, sending...
I20260809 03:32:35.364490 31558 ts_manager.cc:194] Registered new tserver with Master: 278f4ff7f54d458ab0a4a5d3fce58901 (127.27.40.131:36199)
I20260809 03:32:35.364948 31558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:36153
I20260809 03:32:35.368183 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 03:32:35.381639 31627 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:32:35.380509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52246 (local address 127.27.40.190:34123)
0809 03:32:35.380582 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:35.380593 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:35.380732 (+ 139us) server_negotiation.cc:407] Connection header received
0809 03:32:35.380791 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:35.380791 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:35.380801 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:35.380858 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:35.381509 (+ 651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52246: BlockingRecv error: recv got EOF from 127.0.0.1:52246 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 03:32:35.382467 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 31649
I20260809 03:32:35.398018 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 31864
I20260809 03:32:35.410211 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 32068
I20260809 03:32:35.418838 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 31509
2026-08-09T03:32:35Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2087 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:32:35 dist-test-slave-gkw7 krb5kdc[32269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:32:35 dist-test-slave-gkw7 krb5kdc[32269](info): set up 2 sockets
Aug 09 03:32:35 dist-test-slave-gkw7 krb5kdc[32269](info): commencing operation
krb5kdc: starting...
W20260809 03:32:36.640434 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:32:36 dist-test-slave-gkw7 krb5kdc[32269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:32:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:32:36.768337 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:38105
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:38105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:36.849849 32284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:36.850106 32284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:36.850147 32284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:36.851300 32284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:36.851372 32284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:36.851397 32284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:36.851416 32284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:36.851447 32284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:32:36.852947 32284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:38105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:38105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:36.853240 32284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:36.853484 32284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:36.858343 32291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:36.858382 32290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:36.858395 32295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:36.858603 32284 server_base.cc:1034] running on GCE node
I20260809 03:32:36.858893 32284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:36.859213 32284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:36.860390 32284 hybrid_clock.cc:648] HybridClock initialized: now 1786246356860366 us; error 43 us; skew 500 ppm
Aug 09 03:32:36 dist-test-slave-gkw7 krb5kdc[32269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246356, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:36.867995 32284 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:32:36.868547 32284 webserver.cc:460] Webserver started at http://127.27.40.190:45825/ using document root <none> and password file <none>
I20260809 03:32:36.868813 32284 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:36.868887 32284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:36.869067 32284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:36.870247 32284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "65f4d0874e0f4f6db74c08300974609f"
format_stamp: "Formatted at 2026-08-09 03:32:36 on dist-test-slave-gkw7"
server_key: "04fe40ba758f95e7830bed83bf042075"
server_key_iv: "1c2cf279d1da638007bdfd498fcf3d15"
server_key_version: "encryptionkey@0"
I20260809 03:32:36.870612 32284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "65f4d0874e0f4f6db74c08300974609f"
format_stamp: "Formatted at 2026-08-09 03:32:36 on dist-test-slave-gkw7"
server_key: "04fe40ba758f95e7830bed83bf042075"
server_key_iv: "1c2cf279d1da638007bdfd498fcf3d15"
server_key_version: "encryptionkey@0"
I20260809 03:32:36.871984 32284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:32:36.872757 32307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:36.872968 32284 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:36.873075 32284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "65f4d0874e0f4f6db74c08300974609f"
format_stamp: "Formatted at 2026-08-09 03:32:36 on dist-test-slave-gkw7"
server_key: "04fe40ba758f95e7830bed83bf042075"
server_key_iv: "1c2cf279d1da638007bdfd498fcf3d15"
server_key_version: "encryptionkey@0"
I20260809 03:32:36.873179 32284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:36.900641 32284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:36.902909 32284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:36.903131 32284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:36.907720 32284 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:38105
I20260809 03:32:36.907754 32384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:38105 every 8 connection(s)
I20260809 03:32:36.908373 32284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:36.909078 32385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:36.911419 32385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f: Bootstrap starting.
I20260809 03:32:36.911983 32385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:36.912225 32385 log.cc:824] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:36.912878 32385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f: No bootstrap required, opened a new log
I20260809 03:32:36.913630 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 32284
I20260809 03:32:36.913751 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:36.913959 27810 external_mini_cluster.cc:1428] Setting key 2ed46a905fa5bfcda921c7a9952e0a5f
I20260809 03:32:36.915513 32385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f4d0874e0f4f6db74c08300974609f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38105 } }
I20260809 03:32:36.915697 32385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:36.915740 32385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65f4d0874e0f4f6db74c08300974609f, State: Initialized, Role: FOLLOWER
I20260809 03:32:36.915905 32385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f4d0874e0f4f6db74c08300974609f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38105 } }
I20260809 03:32:36.915992 32385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:36.916014 32385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:36.916066 32385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:36.916688 32385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f4d0874e0f4f6db74c08300974609f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38105 } }
I20260809 03:32:36.916795 32385 leader_election.cc:302] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65f4d0874e0f4f6db74c08300974609f; no voters:
I20260809 03:32:36.916945 32385 leader_election.cc:288] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:36.917034 32391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:36.917228 32385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:36.917294 32391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [term 1 LEADER]: Becoming Leader. State: Replica: 65f4d0874e0f4f6db74c08300974609f, State: Running, Role: LEADER
I20260809 03:32:36.917400 32391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f4d0874e0f4f6db74c08300974609f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38105 } }
I20260809 03:32:36.917853 32391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65f4d0874e0f4f6db74c08300974609f. Latest consensus state: current_term: 1 leader_uuid: "65f4d0874e0f4f6db74c08300974609f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f4d0874e0f4f6db74c08300974609f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38105 } } }
I20260809 03:32:36.917977 32391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:36.917866 32393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65f4d0874e0f4f6db74c08300974609f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f4d0874e0f4f6db74c08300974609f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38105 } } }
I20260809 03:32:36.918099 32393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:36.918227 32406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:36.918771 32406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:36.920441 32406 catalog_manager.cc:1347] Generated new cluster ID: 2584216235004ab4a5aa6c5220c1a79f
I20260809 03:32:36.920496 32406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 03:32:36 dist-test-slave-gkw7 krb5kdc[32269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:36.926971 32390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:36.914531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43376 (local address 127.27.40.190:38105)
0809 03:32:36.914746 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:36.914758 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:36.914796 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:32:36.915471 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:36.915480 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:36.915545 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:36.915699 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:36.916538 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:36.917980 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:36.919138 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:36.919648 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:36.919758 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:36.922686 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:36.922690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:36.922691 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:36.922704 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:36.924917 (+ 2213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:36.925314 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:36.925318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:36.925318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:36.925452 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:36.925686 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:36.925687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:36.925687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:36.926042 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:36.926239 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:36.926315 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 03:32:36.926467 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
I20260809 03:32:36.927958 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:32:36.947538 32406 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:36.948261 32406 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:36.949601 32390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:36.944743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43384 (local address 127.27.40.190:38105)
0809 03:32:36.944784 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:36.944786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:36.944960 (+ 174us) server_negotiation.cc:407] Connection header received
0809 03:32:36.944999 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:36.945001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:36.945016 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:36.945088 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:36.945500 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:36.946686 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:36.947682 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:36.947994 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:36.948035 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:36.949000 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:36.949002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:36.949003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:32:36.949252 (+ 249us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:36.949273 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:32:36.949282 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:36.949351 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:36.949408 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 03:32:36.949456 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":186,"server-negotiator.queue_time_us":20}
I20260809 03:32:36.952869 32406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65f4d0874e0f4f6db74c08300974609f: Generated new TSK 0
I20260809 03:32:36.953056 32406 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:32:36.960675 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 32284
2026-08-09T03:32:36Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1556 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:32:37 dist-test-slave-gkw7 krb5kdc[32437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:32:37 dist-test-slave-gkw7 krb5kdc[32437](info): set up 2 sockets
Aug 09 03:32:37 dist-test-slave-gkw7 krb5kdc[32437](info): commencing operation
krb5kdc: starting...
W20260809 03:32:38.252650 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:32:38 dist-test-slave-gkw7 krb5kdc[32437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:32:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:32:38.380457 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:38703
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:38703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:38.463116 32453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:38.463330 32453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:38.463351 32453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:38.464428 32453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:38.464460 32453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:38.464474 32453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:38.464486 32453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:38.464498 32453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:32:38.465938 32453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:38703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:38703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:38.466132 32453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:38.466297 32453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:38.471094 32460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:38.471171 32459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:38.471271 32464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:38.471730 32453 server_base.cc:1034] running on GCE node
I20260809 03:32:38.471987 32453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:38.472332 32453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:38.473598 32453 hybrid_clock.cc:648] HybridClock initialized: now 1786246358473578 us; error 35 us; skew 500 ppm
Aug 09 03:32:38 dist-test-slave-gkw7 krb5kdc[32437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246358, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:38.476094 32453 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:32:38.476603 32453 webserver.cc:460] Webserver started at http://127.27.40.190:36553/ using document root <none> and password file <none>
I20260809 03:32:38.476851 32453 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:38.476907 32453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:38.477041 32453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:38.478160 32453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "7e94d28c1b7d4977bd61b237ea56aee2"
format_stamp: "Formatted at 2026-08-09 03:32:38 on dist-test-slave-gkw7"
server_key: "0925472eecef9b448ad7998529e1d77a"
server_key_iv: "1f30b0592d6a5c3c8eff3f094770f9a0"
server_key_version: "encryptionkey@0"
I20260809 03:32:38.478523 32453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "7e94d28c1b7d4977bd61b237ea56aee2"
format_stamp: "Formatted at 2026-08-09 03:32:38 on dist-test-slave-gkw7"
server_key: "0925472eecef9b448ad7998529e1d77a"
server_key_iv: "1f30b0592d6a5c3c8eff3f094770f9a0"
server_key_version: "encryptionkey@0"
I20260809 03:32:38.479880 32453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:32:38.480670 32475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:38.480965 32453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:38.481062 32453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "7e94d28c1b7d4977bd61b237ea56aee2"
format_stamp: "Formatted at 2026-08-09 03:32:38 on dist-test-slave-gkw7"
server_key: "0925472eecef9b448ad7998529e1d77a"
server_key_iv: "1f30b0592d6a5c3c8eff3f094770f9a0"
server_key_version: "encryptionkey@0"
I20260809 03:32:38.481138 32453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:38.504864 32453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:38.506946 32453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:38.507130 32453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:38.511637 32453 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:38703
I20260809 03:32:38.511677 32560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:38703 every 8 connection(s)
I20260809 03:32:38.512053 32453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:38.513149 32561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:38.515475 32561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2: Bootstrap starting.
I20260809 03:32:38.515653 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 32453
I20260809 03:32:38.515776 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:38.515973 27810 external_mini_cluster.cc:1428] Setting key 230f6d04c6c5b16ea0fdb3af03cbfd50
I20260809 03:32:38.516278 32561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:38.516631 32561 log.cc:824] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:38.517412 32561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2: No bootstrap required, opened a new log
I20260809 03:32:38.519750 32561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e94d28c1b7d4977bd61b237ea56aee2" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38703 } }
I20260809 03:32:38.519923 32561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:38.519958 32561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e94d28c1b7d4977bd61b237ea56aee2, State: Initialized, Role: FOLLOWER
I20260809 03:32:38.520074 32561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e94d28c1b7d4977bd61b237ea56aee2" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38703 } }
I20260809 03:32:38.520165 32561 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:38.520195 32561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:38.520238 32561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:38.520862 32561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e94d28c1b7d4977bd61b237ea56aee2" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38703 } }
I20260809 03:32:38.520953 32561 leader_election.cc:302] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e94d28c1b7d4977bd61b237ea56aee2; no voters:
I20260809 03:32:38.521168 32561 leader_election.cc:288] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:38.521274 32568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:38.521517 32561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:38.521518 32568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [term 1 LEADER]: Becoming Leader. State: Replica: 7e94d28c1b7d4977bd61b237ea56aee2, State: Running, Role: LEADER
I20260809 03:32:38.523464 32568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e94d28c1b7d4977bd61b237ea56aee2" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38703 } }
Aug 09 03:32:38 dist-test-slave-gkw7 krb5kdc[32437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:38.523922 32571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e94d28c1b7d4977bd61b237ea56aee2. Latest consensus state: current_term: 1 leader_uuid: "7e94d28c1b7d4977bd61b237ea56aee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e94d28c1b7d4977bd61b237ea56aee2" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38703 } } }
I20260809 03:32:38.524015 32571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:38.524245 32596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:38.524778 32596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:38.524961 32572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e94d28c1b7d4977bd61b237ea56aee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e94d28c1b7d4977bd61b237ea56aee2" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38703 } } }
I20260809 03:32:38.525296 32572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2 [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:38.528117 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:38.516626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49060 (local address 127.27.40.190:38703)
0809 03:32:38.516843 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:38.516869 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:38.516906 (+ 37us) server_negotiation.cc:407] Connection header received
0809 03:32:38.517467 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:38.517476 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:38.517583 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:38.517721 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:38.518528 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:38.519403 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:38.520497 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:38.520901 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:38.521034 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:38.524473 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:38.524477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:38.524477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:38.524489 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:38.526496 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:38.526946 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:38.526948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:38.526948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:38.527025 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:38.527264 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:38.527264 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:38.527265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:38.527492 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:38.527585 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:38.527797 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 03:32:38.527897 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":85,"threads_started":1}
I20260809 03:32:38.530019 32596 catalog_manager.cc:1347] Generated new cluster ID: 1f958969d8204864807ef98a6b230db4
I20260809 03:32:38.530063 32596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:32:38.557972 32596 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:38.558600 32596 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:38.561120 32596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e94d28c1b7d4977bd61b237ea56aee2: Generated new TSK 0
I20260809 03:32:38.561326 32596 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:32:38.596082 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38703
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:38.680516 32609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:38.680776 32609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:38.680806 32609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:38.682067 32609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:38.682109 32609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:38.682137 32609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:32:38.683568 32609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:38.683794 32609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:38.684000 32609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:38.689137 32621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:38.689088 32618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:38.689079 32617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:38.689360 32609 server_base.cc:1034] running on GCE node
I20260809 03:32:38.689604 32609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:38.689890 32609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:38.691062 32609 hybrid_clock.cc:648] HybridClock initialized: now 1786246358691048 us; error 29 us; skew 500 ppm
Aug 09 03:32:38 dist-test-slave-gkw7 krb5kdc[32437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246358, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:38.693709 32609 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:32:38.694167 32609 webserver.cc:460] Webserver started at http://127.27.40.129:34529/ using document root <none> and password file <none>
I20260809 03:32:38.694375 32609 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:38.694415 32609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:38.694504 32609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:38.695508 32609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "e93f29705ae44675af592da62a0f65fa"
format_stamp: "Formatted at 2026-08-09 03:32:38 on dist-test-slave-gkw7"
server_key: "fd425a2bb6b92ef6669fe65b67d9cf71"
server_key_iv: "f7a05a30ed15698040aea62ac3b7d26d"
server_key_version: "encryptionkey@0"
I20260809 03:32:38.695833 32609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "e93f29705ae44675af592da62a0f65fa"
format_stamp: "Formatted at 2026-08-09 03:32:38 on dist-test-slave-gkw7"
server_key: "fd425a2bb6b92ef6669fe65b67d9cf71"
server_key_iv: "f7a05a30ed15698040aea62ac3b7d26d"
server_key_version: "encryptionkey@0"
I20260809 03:32:38.697118 32609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:32:38.697926 32631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:38.698167 32609 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:32:38.698252 32609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "e93f29705ae44675af592da62a0f65fa"
format_stamp: "Formatted at 2026-08-09 03:32:38 on dist-test-slave-gkw7"
server_key: "fd425a2bb6b92ef6669fe65b67d9cf71"
server_key_iv: "f7a05a30ed15698040aea62ac3b7d26d"
server_key_version: "encryptionkey@0"
I20260809 03:32:38.698351 32609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:38.720978 32609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:38.723692 32609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:38.723870 32609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:38.724117 32609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:38.724462 32609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:38.724547 32609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:38.724596 32609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:38.724629 32609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:38.731007 32609 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:34447
I20260809 03:32:38.731040 331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:34447 every 8 connection(s)
I20260809 03:32:38.731432 32609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
Aug 09 03:32:38 dist-test-slave-gkw7 krb5kdc[32437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246358, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:38.740856 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:38.732249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:42989 (local address 127.27.40.190:38703)
0809 03:32:38.732322 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:38.732324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:38.732547 (+ 223us) server_negotiation.cc:407] Connection header received
0809 03:32:38.732986 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:38.732988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:38.732999 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:38.733062 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:38.734089 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:38.734592 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:38.735670 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:38.735886 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:38.735942 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:38.738319 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:38.738321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:38.738321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:38.738329 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:38.739656 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:38.740027 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:38.740028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:38.740028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:38.740101 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:38.740321 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:38.740322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:38.740322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:38.740425 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:38.740474 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:38.740725 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 03:32:38.740791 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:32:38.741009 336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:38.732209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:38703 (local address 127.27.40.129:42989)
0809 03:32:38.732380 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:38.732421 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:38.732899 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:38.733120 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:38.733123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:38.733757 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:38.733950 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:38.733954 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:38.734700 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:38.734701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:38.735571 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:38.735574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:38.735968 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:38.735969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:38.736102 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:38.736638 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:38.736647 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:38.738220 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:38.739736 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:38.739743 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:38.739745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:38.739961 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:38.740162 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:38.740163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:38.740163 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:38.740265 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:38.740482 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:38.740488 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:38.740607 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 03:32:38.740691 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 03:32:38.740805 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260809 03:32:38.741576 334 heartbeater.cc:347] Connected to a master server at 127.27.40.190:38703
I20260809 03:32:38.741689 334 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:38.741703 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 32609
I20260809 03:32:38.741819 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:32:38.741909 334 heartbeater.cc:511] Master 127.27.40.190:38703 requested a full tablet report, sending...
I20260809 03:32:38.742040 27810 external_mini_cluster.cc:1428] Setting key d76870019c9304dc4cb5cc714df3e55b
I20260809 03:32:38.742553 32505 ts_manager.cc:194] Registered new tserver with Master: e93f29705ae44675af592da62a0f65fa (127.27.40.129:34447)
I20260809 03:32:38.743418 32505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:42989
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:32:38.768754 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38703
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:38.850021 339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:38.850219 339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:38.850241 339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:38.851301 339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:38.851336 339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:38.851361 339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:32:38.852731 339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:38.852921 339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:38.853089 339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:38.858172 347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:38.858244 351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:38.858179 349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:38.858476 339 server_base.cc:1034] running on GCE node
I20260809 03:32:38.858713 339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:38.858984 339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:38.860178 339 hybrid_clock.cc:648] HybridClock initialized: now 1786246358860162 us; error 33 us; skew 500 ppm
Aug 09 03:32:38 dist-test-slave-gkw7 krb5kdc[32437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246358, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:38.863060 339 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:32:38.863595 339 webserver.cc:460] Webserver started at http://127.27.40.130:37149/ using document root <none> and password file <none>
I20260809 03:32:38.863844 339 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:38.863914 339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:38.864066 339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:38.865157 339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "d00b760c084f45daa5cea22f773a3936"
format_stamp: "Formatted at 2026-08-09 03:32:38 on dist-test-slave-gkw7"
server_key: "cbd3dcf91bfd3a7eaeb793f300edd628"
server_key_iv: "1280d488182a851ca818e0da7a97b48d"
server_key_version: "encryptionkey@0"
I20260809 03:32:38.865566 339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "d00b760c084f45daa5cea22f773a3936"
format_stamp: "Formatted at 2026-08-09 03:32:38 on dist-test-slave-gkw7"
server_key: "cbd3dcf91bfd3a7eaeb793f300edd628"
server_key_iv: "1280d488182a851ca818e0da7a97b48d"
server_key_version: "encryptionkey@0"
I20260809 03:32:38.866927 339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:38.867738 363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:38.867939 339 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:38.868034 339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "d00b760c084f45daa5cea22f773a3936"
format_stamp: "Formatted at 2026-08-09 03:32:38 on dist-test-slave-gkw7"
server_key: "cbd3dcf91bfd3a7eaeb793f300edd628"
server_key_iv: "1280d488182a851ca818e0da7a97b48d"
server_key_version: "encryptionkey@0"
I20260809 03:32:38.868090 339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:38.884403 339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:38.886631 339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:38.886775 339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:38.887024 339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:38.887357 339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:38.887392 339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:38.887415 339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:38.887463 339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:38.893489 339 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:36251
I20260809 03:32:38.893561 536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:36251 every 8 connection(s)
I20260809 03:32:38.893887 339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
Aug 09 03:32:38 dist-test-slave-gkw7 krb5kdc[32437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246358, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:38.903172 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:38.894620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:52733 (local address 127.27.40.190:38703)
0809 03:32:38.894691 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:38.894693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:38.894895 (+ 202us) server_negotiation.cc:407] Connection header received
0809 03:32:38.895404 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:38.895405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:38.895416 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:38.895479 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:38.896507 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:38.897005 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:38.898342 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:38.898527 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:38.898575 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:38.900648 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:38.900650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:38.900650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:38.900656 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:38.901956 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:38.902371 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:38.902372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:38.902372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:38.902426 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:38.902648 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:38.902649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:38.902649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:38.902757 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:38.902809 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:38.903034 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 03:32:38.903102 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:32:38.903349 539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:38.894557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:38703 (local address 127.27.40.130:52733)
0809 03:32:38.894747 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:38.894793 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:38.895300 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:38.895599 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:38.895602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:38.896209 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:38.896408 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:38.896413 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:38.897134 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:38.897135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:38.898260 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:38.898263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:38.898597 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:38.898598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:38.898709 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:38.899192 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:38.899200 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:38.900564 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:38.902047 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:38.902055 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:38.902057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:38.902304 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:38.902487 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:38.902487 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:38.902487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:38.902593 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:38.902812 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:38.902816 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:38.902926 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 03:32:38.903008 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 03:32:38.903125 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260809 03:32:38.903837 537 heartbeater.cc:347] Connected to a master server at 127.27.40.190:38703
I20260809 03:32:38.903949 537 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:38.904166 537 heartbeater.cc:511] Master 127.27.40.190:38703 requested a full tablet report, sending...
I20260809 03:32:38.904157 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 339
I20260809 03:32:38.904266 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:32:38.904500 27810 external_mini_cluster.cc:1428] Setting key e1f9f6d331d71054849db9d92ac7fc02
I20260809 03:32:38.904623 32505 ts_manager.cc:194] Registered new tserver with Master: d00b760c084f45daa5cea22f773a3936 (127.27.40.130:36251)
I20260809 03:32:38.905112 32505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:52733
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:32:38.931929 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38703
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:39.014403 542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:39.014606 542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:39.014629 542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:39.015823 542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:39.015870 542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:39.015897 542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:32:39.017356 542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:39.017613 542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:39.017854 542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:39.023139 549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:39.023250 553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:39.023361 551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:39.023388 542 server_base.cc:1034] running on GCE node
I20260809 03:32:39.023660 542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:39.023871 542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:39.025028 542 hybrid_clock.cc:648] HybridClock initialized: now 1786246359025010 us; error 34 us; skew 500 ppm
Aug 09 03:32:39 dist-test-slave-gkw7 krb5kdc[32437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246359, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:39.027873 542 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:32:39.028381 542 webserver.cc:460] Webserver started at http://127.27.40.131:36285/ using document root <none> and password file <none>
I20260809 03:32:39.028637 542 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:39.028715 542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:39.028874 542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:39.030088 542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "4f34d927ce464b4a8b87ca98478f9325"
format_stamp: "Formatted at 2026-08-09 03:32:39 on dist-test-slave-gkw7"
server_key: "3628e90dab2aca9b67b44ce30962a0bf"
server_key_iv: "f4b1e9c0f115a3376c7017036186bed3"
server_key_version: "encryptionkey@0"
I20260809 03:32:39.030488 542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "4f34d927ce464b4a8b87ca98478f9325"
format_stamp: "Formatted at 2026-08-09 03:32:39 on dist-test-slave-gkw7"
server_key: "3628e90dab2aca9b67b44ce30962a0bf"
server_key_iv: "f4b1e9c0f115a3376c7017036186bed3"
server_key_version: "encryptionkey@0"
I20260809 03:32:39.031940 542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:32:39.032737 564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:39.032912 542 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:39.033025 542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "4f34d927ce464b4a8b87ca98478f9325"
format_stamp: "Formatted at 2026-08-09 03:32:39 on dist-test-slave-gkw7"
server_key: "3628e90dab2aca9b67b44ce30962a0bf"
server_key_iv: "f4b1e9c0f115a3376c7017036186bed3"
server_key_version: "encryptionkey@0"
I20260809 03:32:39.033118 542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:39.041393 542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:39.043937 542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:39.044112 542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:39.044361 542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:39.044695 542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:39.044755 542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:39.044801 542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:39.044828 542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:39.051358 542 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:44549
I20260809 03:32:39.051435 727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:44549 every 8 connection(s)
I20260809 03:32:39.051818 542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:32:39.057065 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 542
I20260809 03:32:39.057147 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:32:39.057375 27810 external_mini_cluster.cc:1428] Setting key 1c02c3278100e0b14d9e66c923488a95
Aug 09 03:32:39 dist-test-slave-gkw7 krb5kdc[32437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246359, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:39.063154 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:39.052636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:37621 (local address 127.27.40.190:38703)
0809 03:32:39.052694 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:39.052696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:39.052925 (+ 229us) server_negotiation.cc:407] Connection header received
0809 03:32:39.053698 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:39.053700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:39.053712 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:39.053778 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:39.055071 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:39.055649 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:39.056886 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:39.057094 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:39.057229 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:39.060002 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:39.060003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:39.060003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:39.060010 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:39.061763 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:39.062182 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:39.062183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:39.062184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:39.062240 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:39.062523 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:39.062523 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:39.062523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:39.062636 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:39.062693 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:39.062985 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 03:32:39.063079 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 03:32:39.063374 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:39.052606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:38703 (local address 127.27.40.131:37621)
0809 03:32:39.052781 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:39.052823 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:39.053549 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:39.053886 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:39.053890 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:39.054666 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:39.054939 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:39.054945 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:39.055754 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:39.055755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:39.056788 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:39.056791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:39.057274 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:39.057276 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:39.057433 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:39.058136 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:39.058148 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:39.059909 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:39.061854 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:39.061861 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:39.061863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:39.062108 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:39.062327 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:39.062328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:39.062329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:39.062451 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:39.062727 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:39.062734 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:39.062857 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 03:32:39.062957 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 03:32:39.063094 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260809 03:32:39.063920 728 heartbeater.cc:347] Connected to a master server at 127.27.40.190:38703
I20260809 03:32:39.064044 728 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:39.064244 728 heartbeater.cc:511] Master 127.27.40.190:38703 requested a full tablet report, sending...
I20260809 03:32:39.064670 32505 ts_manager.cc:194] Registered new tserver with Master: 4f34d927ce464b4a8b87ca98478f9325 (127.27.40.131:44549)
I20260809 03:32:39.065039 32505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:37621
I20260809 03:32:39.069744 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 03:32:39 dist-test-slave-gkw7 krb5kdc[32437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246359, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 03:32:39 dist-test-slave-gkw7 krb5kdc[32437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246359, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:39.097177 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:39.089309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49068 (local address 127.27.40.190:38703)
0809 03:32:39.089382 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:39.089384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:39.089495 (+ 111us) server_negotiation.cc:407] Connection header received
0809 03:32:39.089605 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:39.089606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:39.089617 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:39.089683 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:39.090332 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:39.090842 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:39.091687 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:39.091871 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:39.091929 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:39.094444 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:39.094445 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:39.094446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:39.094452 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:39.095963 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:39.096357 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:39.096358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:39.096358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:39.096416 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:39.096625 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:39.096626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:39.096626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:39.096816 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:39.096914 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:39.097009 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 03:32:39.097077 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 03:32:39.097445 32505 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49068
I20260809 03:32:39.097939 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 32609
I20260809 03:32:39.112953 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 339
I20260809 03:32:39.130105 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 542
I20260809 03:32:39.140467 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 32453
2026-08-09T03:32:39Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2164 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:32:39 dist-test-slave-gkw7 krb5kdc[749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:32:39 dist-test-slave-gkw7 krb5kdc[749](info): set up 2 sockets
Aug 09 03:32:39 dist-test-slave-gkw7 krb5kdc[749](info): commencing operation
krb5kdc: starting...
W20260809 03:32:40.386518 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:32:40 dist-test-slave-gkw7 krb5kdc[749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:32:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:32:40.512473 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:36321
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:36721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:36321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:40.596750 766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:40.596952 766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:40.596975 766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:40.598264 766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:40.598326 766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:40.598341 766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:40.598366 766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:40.598377 766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:32:40.599925 766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:36721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:36321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:36321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:40.600181 766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:40.600419 766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:40.605278 774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:40.605273 778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:40.605312 775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:40.605629 766 server_base.cc:1034] running on GCE node
I20260809 03:32:40.605877 766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:40.606213 766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:40.607409 766 hybrid_clock.cc:648] HybridClock initialized: now 1786246360607391 us; error 46 us; skew 500 ppm
Aug 09 03:32:40 dist-test-slave-gkw7 krb5kdc[749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246360, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:40.609987 766 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:32:40.610457 766 webserver.cc:460] Webserver started at http://127.27.40.190:44377/ using document root <none> and password file <none>
I20260809 03:32:40.610711 766 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:40.610781 766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:40.610931 766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:40.612003 766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "be482994bf1e42e1a1d2f6e6d7725a84"
format_stamp: "Formatted at 2026-08-09 03:32:40 on dist-test-slave-gkw7"
server_key: "9972a6f60c4cab9f964d70c35f68ad9f"
server_key_iv: "b70cf4d6a7fb9ea1c62319d1af45476b"
server_key_version: "encryptionkey@0"
I20260809 03:32:40.612416 766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "be482994bf1e42e1a1d2f6e6d7725a84"
format_stamp: "Formatted at 2026-08-09 03:32:40 on dist-test-slave-gkw7"
server_key: "9972a6f60c4cab9f964d70c35f68ad9f"
server_key_iv: "b70cf4d6a7fb9ea1c62319d1af45476b"
server_key_version: "encryptionkey@0"
I20260809 03:32:40.613780 766 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 03:32:40.614485 787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:40.614681 766 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:40.614789 766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "be482994bf1e42e1a1d2f6e6d7725a84"
format_stamp: "Formatted at 2026-08-09 03:32:40 on dist-test-slave-gkw7"
server_key: "9972a6f60c4cab9f964d70c35f68ad9f"
server_key_iv: "b70cf4d6a7fb9ea1c62319d1af45476b"
server_key_version: "encryptionkey@0"
I20260809 03:32:40.614868 766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:40.624571 766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:40.626469 766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:40.626623 766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:40.630654 766 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:36321
I20260809 03:32:40.630692 865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:36321 every 8 connection(s)
I20260809 03:32:40.631210 766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:40.631907 866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:40.634459 866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84: Bootstrap starting.
I20260809 03:32:40.635071 866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:40.635375 866 log.cc:824] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:40.636148 866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84: No bootstrap required, opened a new log
I20260809 03:32:40.637717 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 766
I20260809 03:32:40.637822 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:40.638010 27810 external_mini_cluster.cc:1428] Setting key b3588cdc266681b5bc675ae9754287b5
I20260809 03:32:40.638036 866 raft_consensus.cc:348] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be482994bf1e42e1a1d2f6e6d7725a84" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36321 } }
I20260809 03:32:40.638232 866 raft_consensus.cc:374] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:40.638262 866 raft_consensus.cc:729] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be482994bf1e42e1a1d2f6e6d7725a84, State: Initialized, Role: FOLLOWER
I20260809 03:32:40.638432 866 consensus_queue.cc:260] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be482994bf1e42e1a1d2f6e6d7725a84" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36321 } }
I20260809 03:32:40.638537 866 raft_consensus.cc:388] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:40.638576 866 raft_consensus.cc:482] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:40.638619 866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:40.639164 866 raft_consensus.cc:504] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be482994bf1e42e1a1d2f6e6d7725a84" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36321 } }
I20260809 03:32:40.639287 866 leader_election.cc:302] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be482994bf1e42e1a1d2f6e6d7725a84; no voters:
I20260809 03:32:40.639482 866 leader_election.cc:288] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:40.639509 871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:40.639743 866 sys_catalog.cc:564] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:40.639766 871 raft_consensus.cc:686] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [term 1 LEADER]: Becoming Leader. State: Replica: be482994bf1e42e1a1d2f6e6d7725a84, State: Running, Role: LEADER
I20260809 03:32:40.639886 871 consensus_queue.cc:237] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be482994bf1e42e1a1d2f6e6d7725a84" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36321 } }
I20260809 03:32:40.640342 873 sys_catalog.cc:455] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be482994bf1e42e1a1d2f6e6d7725a84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be482994bf1e42e1a1d2f6e6d7725a84" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36321 } } }
I20260809 03:32:40.640514 873 sys_catalog.cc:458] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:40.640666 875 sys_catalog.cc:455] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be482994bf1e42e1a1d2f6e6d7725a84. Latest consensus state: current_term: 1 leader_uuid: "be482994bf1e42e1a1d2f6e6d7725a84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be482994bf1e42e1a1d2f6e6d7725a84" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36321 } } }
I20260809 03:32:40.640884 875 sys_catalog.cc:458] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84 [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:40.641095 891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:40.641973 891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:40.643364 891 catalog_manager.cc:1347] Generated new cluster ID: 7d089dd4686a48928333c3eeb6f01b47
I20260809 03:32:40.643416 891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 03:32:40 dist-test-slave-gkw7 krb5kdc[749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:40.650955 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:40.638522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40260 (local address 127.27.40.190:36321)
0809 03:32:40.639182 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:40.639191 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:40.639224 (+ 33us) server_negotiation.cc:407] Connection header received
0809 03:32:40.639348 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:40.639355 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:40.639411 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:40.639568 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:40.640392 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:40.641884 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:40.642980 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:40.643581 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:40.643708 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:40.646680 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:40.646684 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:40.646685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:40.646696 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:40.648784 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:40.649227 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:40.649231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:40.649232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:40.649394 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:40.649660 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:40.649661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:40.649661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:40.649985 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:40.650148 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:40.650607 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 03:32:40.650759 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":579,"threads_started":1}
I20260809 03:32:40.651944 891 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:40.652441 891 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:40.660385 891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be482994bf1e42e1a1d2f6e6d7725a84: Generated new TSK 0
I20260809 03:32:40.660569 891 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:32:40.692740 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:36321
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:36721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:40.775480 900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:40.775718 900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:40.775756 900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:40.776886 900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:40.776954 900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:40.776998 900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:32:40.778733 900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:36721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:36321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:40.778990 900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:40.779224 900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:40.785034 906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:40.785050 908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:40.785151 911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:40.785292 900 server_base.cc:1034] running on GCE node
I20260809 03:32:40.785498 900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:40.785785 900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:40.786950 900 hybrid_clock.cc:648] HybridClock initialized: now 1786246360786931 us; error 40 us; skew 500 ppm
Aug 09 03:32:40 dist-test-slave-gkw7 krb5kdc[749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246360, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:40.789682 900 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:32:40.790169 900 webserver.cc:460] Webserver started at http://127.27.40.129:45207/ using document root <none> and password file <none>
I20260809 03:32:40.790405 900 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:40.790472 900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:40.790609 900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:40.791697 900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "950e7707395e49f58eab33122c3e386a"
format_stamp: "Formatted at 2026-08-09 03:32:40 on dist-test-slave-gkw7"
server_key: "7d12ddf946eed3e43961d0900ddb2673"
server_key_iv: "b58a8e17d509d25c810029ad2924f88b"
server_key_version: "encryptionkey@0"
I20260809 03:32:40.792071 900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "950e7707395e49f58eab33122c3e386a"
format_stamp: "Formatted at 2026-08-09 03:32:40 on dist-test-slave-gkw7"
server_key: "7d12ddf946eed3e43961d0900ddb2673"
server_key_iv: "b58a8e17d509d25c810029ad2924f88b"
server_key_version: "encryptionkey@0"
I20260809 03:32:40.793457 900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:32:40.794240 921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:40.794413 900 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:40.794518 900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "950e7707395e49f58eab33122c3e386a"
format_stamp: "Formatted at 2026-08-09 03:32:40 on dist-test-slave-gkw7"
server_key: "7d12ddf946eed3e43961d0900ddb2673"
server_key_iv: "b58a8e17d509d25c810029ad2924f88b"
server_key_version: "encryptionkey@0"
I20260809 03:32:40.794590 900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:40.803141 900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:40.804916 900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:40.805071 900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:40.805298 900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:40.805694 900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:40.805750 900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:40.805791 900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:40.805819 900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:40.811829 900 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:37123
I20260809 03:32:40.811879 1102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:37123 every 8 connection(s)
I20260809 03:32:40.812183 900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
Aug 09 03:32:40 dist-test-slave-gkw7 krb5kdc[749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246360, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:40.818790 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 900
I20260809 03:32:40.818902 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:32:40.819080 27810 external_mini_cluster.cc:1428] Setting key 5738f7d36cc4f9ce134bfaba27f10c59
I20260809 03:32:40.821987 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:40.812905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:33805 (local address 127.27.40.190:36321)
0809 03:32:40.812967 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:40.812969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:40.813150 (+ 181us) server_negotiation.cc:407] Connection header received
0809 03:32:40.814092 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:40.814093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:40.814107 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:40.814168 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:40.814753 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:40.815311 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:40.816316 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:40.816522 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:40.816587 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:40.818688 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:40.818690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:40.818690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:40.818697 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:40.820583 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:40.820959 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:40.820960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:40.820961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:40.821024 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:40.821255 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:40.821255 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:40.821256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:40.821402 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:40.821472 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:40.821805 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 03:32:40.821899 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 03:32:40.822185 1105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:40.812871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:36321 (local address 127.27.40.129:33805)
0809 03:32:40.813024 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:40.813064 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:40.813989 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:40.814221 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:40.814223 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:40.814467 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:40.814658 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:40.814662 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:40.815404 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:40.815405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:40.816221 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:40.816223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:40.816604 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:40.816605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:40.816712 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:40.817181 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:40.817189 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:40.818586 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:40.820667 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:40.820674 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:40.820675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:40.820885 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:40.821091 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:40.821091 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:40.821092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:40.821192 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:40.821466 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:40.821471 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:40.821633 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 03:32:40.821753 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 03:32:40.821923 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260809 03:32:40.822678 1103 heartbeater.cc:347] Connected to a master server at 127.27.40.190:36321
I20260809 03:32:40.822783 1103 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:40.822988 1103 heartbeater.cc:511] Master 127.27.40.190:36321 requested a full tablet report, sending...
I20260809 03:32:40.823519 812 ts_manager.cc:194] Registered new tserver with Master: 950e7707395e49f58eab33122c3e386a (127.27.40.129:37123)
I20260809 03:32:40.824137 812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:33805
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:32:40.845036 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:36321
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:36721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:40.928781 1108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:40.928977 1108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:40.928999 1108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:40.930222 1108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:40.930266 1108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:40.930294 1108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:32:40.931784 1108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:36721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:36321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:40.932024 1108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:40.932260 1108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:40.937227 1121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:40.937281 1108 server_base.cc:1034] running on GCE node
W20260809 03:32:40.937244 1119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:40.937228 1124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:40.937680 1108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:40.937947 1108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:40.939114 1108 hybrid_clock.cc:648] HybridClock initialized: now 1786246360939096 us; error 29 us; skew 500 ppm
Aug 09 03:32:40 dist-test-slave-gkw7 krb5kdc[749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246360, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:40.942090 1108 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:32:40.942543 1108 webserver.cc:460] Webserver started at http://127.27.40.130:34229/ using document root <none> and password file <none>
I20260809 03:32:40.942798 1108 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:40.942870 1108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:40.943027 1108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:40.944064 1108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "9685ef5d5baa444d8bb564951b9d508b"
format_stamp: "Formatted at 2026-08-09 03:32:40 on dist-test-slave-gkw7"
server_key: "7533ac02b04beae50088adc39fa37974"
server_key_iv: "b74c7d7d71ee629c1a306e3361ec0e59"
server_key_version: "encryptionkey@0"
I20260809 03:32:40.944442 1108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "9685ef5d5baa444d8bb564951b9d508b"
format_stamp: "Formatted at 2026-08-09 03:32:40 on dist-test-slave-gkw7"
server_key: "7533ac02b04beae50088adc39fa37974"
server_key_iv: "b74c7d7d71ee629c1a306e3361ec0e59"
server_key_version: "encryptionkey@0"
I20260809 03:32:40.945833 1108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 03:32:40.946589 1140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:40.946775 1108 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:40.946869 1108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "9685ef5d5baa444d8bb564951b9d508b"
format_stamp: "Formatted at 2026-08-09 03:32:40 on dist-test-slave-gkw7"
server_key: "7533ac02b04beae50088adc39fa37974"
server_key_iv: "b74c7d7d71ee629c1a306e3361ec0e59"
server_key_version: "encryptionkey@0"
I20260809 03:32:40.946951 1108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:40.969810 1108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:40.972280 1108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:40.972450 1108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:40.972715 1108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:40.973032 1108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:40.973088 1108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:40.973141 1108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:40.973170 1108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:40.979245 1108 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:36423
I20260809 03:32:40.979301 1323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:36423 every 8 connection(s)
I20260809 03:32:40.979662 1108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:32:40.980589 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 1108
I20260809 03:32:40.980716 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:32:40.980898 27810 external_mini_cluster.cc:1428] Setting key 5f1986289a61c0cf2aa287e9b589535e
Aug 09 03:32:40 dist-test-slave-gkw7 krb5kdc[749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246360, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
I20260809 03:32:40.990744 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:40.980557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:56955 (local address 127.27.40.190:36321)
0809 03:32:40.980614 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:40.980616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:40.980951 (+ 335us) server_negotiation.cc:407] Connection header received
0809 03:32:40.982047 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:40.982048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:40.982061 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:40.982111 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:40.982756 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:40.983274 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:40.984304 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:40.984607 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:40.984682 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:40.987226 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:40.987228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:40.987228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:40.987237 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:40.989006 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:40.989443 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:40.989445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:40.989445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:40.989508 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:40.989833 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:40.989834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:40.989834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:40.989996 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:40.990085 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:40.990510 (+ 425us) server_negotiation.cc:299] Negotiation successful
0809 03:32:40.990608 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 03:32:40.990984 1330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:40.980538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:36321 (local address 127.27.40.130:56955)
0809 03:32:40.980806 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:40.980853 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:40.981931 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:40.982182 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:40.982186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:40.982458 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:40.982657 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:40.982661 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:40.983372 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:40.983373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:40.984207 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:40.984209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:40.984698 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:40.984699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:40.984814 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:40.985373 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:40.985383 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:40.987120 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:40.989119 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:40.989125 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:40.989127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:40.989354 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:40.989653 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:40.989654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:40.989654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:40.989757 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:40.990134 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:40.990141 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:40.990293 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 03:32:40.990471 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 03:32:40.990678 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":109,"threads_started":1}
I20260809 03:32:40.991469 1327 heartbeater.cc:347] Connected to a master server at 127.27.40.190:36321
I20260809 03:32:40.991573 1327 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:40.991773 1327 heartbeater.cc:511] Master 127.27.40.190:36321 requested a full tablet report, sending...
I20260809 03:32:40.992208 812 ts_manager.cc:194] Registered new tserver with Master: 9685ef5d5baa444d8bb564951b9d508b (127.27.40.130:36423)
I20260809 03:32:40.992560 812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:56955
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:32:41.008531 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:36321
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:36721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:41.098558 1337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:41.098809 1337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:41.098850 1337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:41.100020 1337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:41.100139 1337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:41.100188 1337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:32:41.101768 1337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:36721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:36321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:41.102067 1337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:41.102308 1337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:41.107847 1346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:41.107847 1347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:41.107959 1350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:41.108134 1337 server_base.cc:1034] running on GCE node
I20260809 03:32:41.108441 1337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:41.108705 1337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:41.109881 1337 hybrid_clock.cc:648] HybridClock initialized: now 1786246361109884 us; error 47 us; skew 500 ppm
Aug 09 03:32:41 dist-test-slave-gkw7 krb5kdc[749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246361, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:41.112499 1337 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:32:41.113008 1337 webserver.cc:460] Webserver started at http://127.27.40.131:44107/ using document root <none> and password file <none>
I20260809 03:32:41.113269 1337 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:41.113341 1337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:41.113502 1337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:41.114626 1337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "a6082c0d17144c72a8c4033f03a3ead9"
format_stamp: "Formatted at 2026-08-09 03:32:41 on dist-test-slave-gkw7"
server_key: "36a114581bc4bd74c7cb7bbfd051ee87"
server_key_iv: "6ed42e95927856edaf559fb13aaa6d84"
server_key_version: "encryptionkey@0"
I20260809 03:32:41.115003 1337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "a6082c0d17144c72a8c4033f03a3ead9"
format_stamp: "Formatted at 2026-08-09 03:32:41 on dist-test-slave-gkw7"
server_key: "36a114581bc4bd74c7cb7bbfd051ee87"
server_key_iv: "6ed42e95927856edaf559fb13aaa6d84"
server_key_version: "encryptionkey@0"
I20260809 03:32:41.116528 1337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:32:41.117573 1362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:41.117772 1337 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:32:41.117878 1337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "a6082c0d17144c72a8c4033f03a3ead9"
format_stamp: "Formatted at 2026-08-09 03:32:41 on dist-test-slave-gkw7"
server_key: "36a114581bc4bd74c7cb7bbfd051ee87"
server_key_iv: "6ed42e95927856edaf559fb13aaa6d84"
server_key_version: "encryptionkey@0"
I20260809 03:32:41.117964 1337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:41.142014 1337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:41.143985 1337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:41.144166 1337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:41.144419 1337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:41.144748 1337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:41.144805 1337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:41.144848 1337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:41.144882 1337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:41.152282 1337 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:36515
I20260809 03:32:41.152335 1522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:36515 every 8 connection(s)
I20260809 03:32:41.152740 1337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:32:41.154609 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 1337
I20260809 03:32:41.154723 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:32:41.154958 27810 external_mini_cluster.cc:1428] Setting key 1c8b3e7231ee975eede15195fa7bc4ad
Aug 09 03:32:41 dist-test-slave-gkw7 krb5kdc[749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246361, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:41.162796 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:41.153506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:35251 (local address 127.27.40.190:36321)
0809 03:32:41.153604 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:41.153607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:41.153794 (+ 187us) server_negotiation.cc:407] Connection header received
0809 03:32:41.155134 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:41.155135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:41.155150 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:41.155205 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:41.156206 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:41.156712 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:41.157761 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:41.157945 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:41.158000 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:41.160115 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:41.160116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:41.160116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:41.160122 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:41.161650 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:41.162021 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:41.162021 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:41.162022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:41.162074 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:41.162285 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:41.162286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:41.162286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:41.162386 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:41.162438 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:41.162665 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 03:32:41.162731 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 03:32:41.162950 1525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:41.153480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:36321 (local address 127.27.40.131:35251)
0809 03:32:41.153658 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:41.153706 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:41.154993 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:41.155378 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:41.155388 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:41.155863 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:41.156127 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:41.156132 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:41.156794 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:41.156795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:41.157690 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:41.157693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:41.158017 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:41.158018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:41.158118 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:41.158645 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:41.158653 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:41.160033 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:41.161719 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:41.161725 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:41.161727 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:41.161955 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:41.162133 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:41.162134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:41.162134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:41.162230 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:41.162440 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:41.162445 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:41.162555 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 03:32:41.162635 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 03:32:41.162753 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
I20260809 03:32:41.163439 1523 heartbeater.cc:347] Connected to a master server at 127.27.40.190:36321
I20260809 03:32:41.163534 1523 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:41.163681 1523 heartbeater.cc:511] Master 127.27.40.190:36321 requested a full tablet report, sending...
I20260809 03:32:41.164070 812 ts_manager.cc:194] Registered new tserver with Master: a6082c0d17144c72a8c4033f03a3ead9 (127.27.40.131:36515)
I20260809 03:32:41.164446 812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:35251
I20260809 03:32:41.166734 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 03:32:41 dist-test-slave-gkw7 krb5kdc[749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 03:32:41 dist-test-slave-gkw7 krb5kdc[749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.129@KRBTEST.COM
I20260809 03:32:41.192217 1532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:41.183267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36900 (local address 127.27.40.129:37123)
0809 03:32:41.183491 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:41.183497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:41.183506 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:41.183537 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:41.183540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:41.183557 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:41.183681 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:41.184437 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:41.185263 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:41.186199 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:41.186555 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:41.186655 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:41.189186 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:41.189195 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:41.189196 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:41.189203 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:41.190805 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:41.191228 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:41.191231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:41.191232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:41.191315 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:41.191540 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:41.191541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:41.191542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:41.191699 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:41.191881 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:41.191963 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 03:32:41.192064 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":70,"threads_started":1}
Aug 09 03:32:41 dist-test-slave-gkw7 krb5kdc[749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:41.200834 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:41.193630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40270 (local address 127.27.40.190:36321)
0809 03:32:41.193685 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:41.193687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:41.193750 (+ 63us) server_negotiation.cc:407] Connection header received
0809 03:32:41.193787 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:41.193788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:41.193798 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:41.193844 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:41.194305 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:41.194985 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:41.196078 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:41.196355 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:41.196420 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:41.198780 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:41.198781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:41.198782 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:41.198788 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:41.199908 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:41.200204 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:41.200205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:41.200205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:41.200258 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:41.200418 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:41.200419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:41.200419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:41.200551 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:41.200625 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:41.200708 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 03:32:41.200770 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260809 03:32:41.201103 812 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40270
I20260809 03:32:41.201514 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 900
I20260809 03:32:41.210956 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 1108
I20260809 03:32:41.234656 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 1337
I20260809 03:32:41.243876 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 766
2026-08-09T03:32:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2103 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:32:41 dist-test-slave-gkw7 krb5kdc[1539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:32:41 dist-test-slave-gkw7 krb5kdc[1539](info): set up 2 sockets
Aug 09 03:32:41 dist-test-slave-gkw7 krb5kdc[1539](info): commencing operation
krb5kdc: starting...
W20260809 03:32:42.542845 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:32:42 dist-test-slave-gkw7 krb5kdc[1539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:32:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:32:42.667754 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:38693
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:38693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:42.750626 1553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:42.750830 1553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:42.750854 1553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:42.751936 1553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:42.751971 1553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:42.751984 1553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:42.751997 1553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:42.752009 1553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:32:42.753456 1553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:38693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:38693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:42.753676 1553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:42.753842 1553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:42.758775 1560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:42.758850 1553 server_base.cc:1034] running on GCE node
W20260809 03:32:42.758733 1559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:42.759006 1562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:42.759280 1553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:42.759564 1553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:42.760706 1553 hybrid_clock.cc:648] HybridClock initialized: now 1786246362760693 us; error 28 us; skew 500 ppm
Aug 09 03:32:42 dist-test-slave-gkw7 krb5kdc[1539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246362, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:42.767762 1553 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:32:42.768164 1553 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:42.768239 1553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:42.768399 1553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:42.769464 1553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "b9a88f289e274d3ca77026162f245c1b"
format_stamp: "Formatted at 2026-08-09 03:32:42 on dist-test-slave-gkw7"
server_key: "a0573bb5c6d2723eafa61391ec68b16d"
server_key_iv: "265e549069f233d73e67564410ccede0"
server_key_version: "encryptionkey@0"
I20260809 03:32:42.769896 1553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "b9a88f289e274d3ca77026162f245c1b"
format_stamp: "Formatted at 2026-08-09 03:32:42 on dist-test-slave-gkw7"
server_key: "a0573bb5c6d2723eafa61391ec68b16d"
server_key_iv: "265e549069f233d73e67564410ccede0"
server_key_version: "encryptionkey@0"
I20260809 03:32:42.771275 1553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:42.771987 1575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:42.772153 1553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:42.772251 1553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "b9a88f289e274d3ca77026162f245c1b"
format_stamp: "Formatted at 2026-08-09 03:32:42 on dist-test-slave-gkw7"
server_key: "a0573bb5c6d2723eafa61391ec68b16d"
server_key_iv: "265e549069f233d73e67564410ccede0"
server_key_version: "encryptionkey@0"
I20260809 03:32:42.772332 1553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:42.790609 1553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:42.792418 1553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:42.792583 1553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:42.797008 1553 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:38693
I20260809 03:32:42.797036 1652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:38693 every 8 connection(s)
I20260809 03:32:42.797335 1553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:42.798285 1653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:42.800529 1653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b: Bootstrap starting.
I20260809 03:32:42.801074 1653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:42.801307 1653 log.cc:824] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:42.801988 1653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b: No bootstrap required, opened a new log
I20260809 03:32:42.802908 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 1553
I20260809 03:32:42.803020 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:42.803218 27810 external_mini_cluster.cc:1428] Setting key 8a7d119fecf85814858c39bbc6429b47
I20260809 03:32:42.804111 1653 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a88f289e274d3ca77026162f245c1b" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38693 } }
I20260809 03:32:42.804273 1653 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:42.804318 1653 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9a88f289e274d3ca77026162f245c1b, State: Initialized, Role: FOLLOWER
I20260809 03:32:42.804472 1653 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a88f289e274d3ca77026162f245c1b" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38693 } }
I20260809 03:32:42.804566 1653 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:42.804603 1653 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:42.804668 1653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:42.805327 1653 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a88f289e274d3ca77026162f245c1b" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38693 } }
I20260809 03:32:42.805414 1653 leader_election.cc:302] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9a88f289e274d3ca77026162f245c1b; no voters:
I20260809 03:32:42.805630 1653 leader_election.cc:288] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:42.805671 1661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:42.805799 1661 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [term 1 LEADER]: Becoming Leader. State: Replica: b9a88f289e274d3ca77026162f245c1b, State: Running, Role: LEADER
I20260809 03:32:42.805883 1661 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a88f289e274d3ca77026162f245c1b" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38693 } }
I20260809 03:32:42.805912 1653 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:42.806200 1661 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9a88f289e274d3ca77026162f245c1b. Latest consensus state: current_term: 1 leader_uuid: "b9a88f289e274d3ca77026162f245c1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a88f289e274d3ca77026162f245c1b" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38693 } } }
I20260809 03:32:42.806272 1661 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:42.806210 1662 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9a88f289e274d3ca77026162f245c1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a88f289e274d3ca77026162f245c1b" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38693 } } }
I20260809 03:32:42.806504 1662 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:42.807307 1667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:42.807925 1667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:42.809576 1667 catalog_manager.cc:1347] Generated new cluster ID: e6220a1663e64e15b688be72924ad409
I20260809 03:32:42.809631 1667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 03:32:42 dist-test-slave-gkw7 krb5kdc[1539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:42.814968 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:42.803778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40536 (local address 127.27.40.190:38693)
0809 03:32:42.803965 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:42.803973 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:42.803998 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:32:42.804125 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:42.804130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:42.804172 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:42.804318 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:42.805100 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:42.806562 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:42.807510 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:42.808216 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:42.808328 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:42.811271 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:42.811275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:42.811276 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:42.811287 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:42.813200 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:42.813650 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:42.813653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:42.813653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:42.813764 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:42.813989 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:42.813989 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:42.813990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:42.814265 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:42.814412 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:42.814572 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 03:32:42.814707 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260809 03:32:42.839814 1667 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:42.840355 1667 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:42.843322 1667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b: Generated new TSK 0
I20260809 03:32:42.843513 1667 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:32:42.877099 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38693
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:42.961696 1690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:42.961969 1690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:42.962000 1690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:42.963163 1690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:42.963204 1690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:42.963263 1690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:32:42.964762 1690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:42.965006 1690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:42.965243 1690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:42.970428 1696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:42.970460 1695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:42.970496 1690 server_base.cc:1034] running on GCE node
W20260809 03:32:42.970456 1699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:42.970866 1690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:42.971060 1690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:42.972194 1690 hybrid_clock.cc:648] HybridClock initialized: now 1786246362972167 us; error 46 us; skew 500 ppm
Aug 09 03:32:42 dist-test-slave-gkw7 krb5kdc[1539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246362, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:42.974766 1690 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:32:42.975209 1690 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:42.975261 1690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:42.975448 1690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:42.976562 1690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "16097ae5301b4f7eb8cfc5efc0bdfec3"
format_stamp: "Formatted at 2026-08-09 03:32:42 on dist-test-slave-gkw7"
server_key: "2695d1fd55b7e6b7bcdced3e1063e724"
server_key_iv: "4c14b0601d4c4dbff794ad366de6bcd7"
server_key_version: "encryptionkey@0"
I20260809 03:32:42.976920 1690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "16097ae5301b4f7eb8cfc5efc0bdfec3"
format_stamp: "Formatted at 2026-08-09 03:32:42 on dist-test-slave-gkw7"
server_key: "2695d1fd55b7e6b7bcdced3e1063e724"
server_key_iv: "4c14b0601d4c4dbff794ad366de6bcd7"
server_key_version: "encryptionkey@0"
I20260809 03:32:42.978335 1690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:32:42.979147 1706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:42.979326 1690 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:42.979417 1690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "16097ae5301b4f7eb8cfc5efc0bdfec3"
format_stamp: "Formatted at 2026-08-09 03:32:42 on dist-test-slave-gkw7"
server_key: "2695d1fd55b7e6b7bcdced3e1063e724"
server_key_iv: "4c14b0601d4c4dbff794ad366de6bcd7"
server_key_version: "encryptionkey@0"
I20260809 03:32:42.979496 1690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:42.995441 1690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:42.997473 1690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:42.997720 1690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:42.997972 1690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:42.998304 1690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:42.998360 1690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:42.998395 1690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:42.998425 1690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:43.004310 1690 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:36749
I20260809 03:32:43.004357 1871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:36749 every 8 connection(s)
I20260809 03:32:43.004617 1690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
Aug 09 03:32:43 dist-test-slave-gkw7 krb5kdc[1539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246362, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:43.012724 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 1690
I20260809 03:32:43.012871 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:32:43.013062 27810 external_mini_cluster.cc:1428] Setting key 0cbffbd77f9dcc9d96f6c7143a49cd0e
I20260809 03:32:43.014629 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.005324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:37141 (local address 127.27.40.190:38693)
0809 03:32:43.005392 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:43.005394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:43.005620 (+ 226us) server_negotiation.cc:407] Connection header received
0809 03:32:43.006257 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:43.006258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:43.006272 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:43.006335 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:43.007377 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.007895 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.008895 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.009143 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.009218 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.011450 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:43.011452 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:43.011452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:43.011460 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:43.013061 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:43.013593 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:43.013595 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:43.013595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:43.013662 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:43.013936 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:43.013937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:43.013937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:43.014057 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:43.014138 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:43.014430 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 03:32:43.014537 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 03:32:43.014787 1874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.005297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:38693 (local address 127.27.40.129:37141)
0809 03:32:43.005483 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:43.005547 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:43.006171 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:43.006390 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:43.006393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:43.006648 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:43.007288 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:43.007293 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.007971 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.007972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:43.008796 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:43.008798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.009256 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.009257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:43.009377 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.009887 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:43.009895 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:43.011345 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:43.013174 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:43.013182 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:43.013184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:43.013481 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:43.013733 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:43.013734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:43.013734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:43.013874 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:43.014135 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:43.014143 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:43.014282 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 03:32:43.014381 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 03:32:43.014548 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":106,"threads_started":1}
I20260809 03:32:43.015498 1872 heartbeater.cc:347] Connected to a master server at 127.27.40.190:38693
I20260809 03:32:43.015611 1872 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:43.015765 1872 heartbeater.cc:511] Master 127.27.40.190:38693 requested a full tablet report, sending...
I20260809 03:32:43.016352 1597 ts_manager.cc:194] Registered new tserver with Master: 16097ae5301b4f7eb8cfc5efc0bdfec3 (127.27.40.129:36749)
I20260809 03:32:43.017392 1597 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:37141
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:32:43.040654 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38693
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:43.124573 1877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:43.124781 1877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:43.124806 1877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:43.125976 1877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:43.126016 1877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:43.126042 1877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:32:43.127496 1877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:43.127669 1877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:43.127857 1877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:43.132866 1886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:43.132862 1887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:43.132863 1889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:43.133019 1877 server_base.cc:1034] running on GCE node
I20260809 03:32:43.133293 1877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:43.133577 1877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:43.134758 1877 hybrid_clock.cc:648] HybridClock initialized: now 1786246363134748 us; error 27 us; skew 500 ppm
Aug 09 03:32:43 dist-test-slave-gkw7 krb5kdc[1539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246363, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:43.137362 1877 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:32:43.137800 1877 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:43.137873 1877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:43.138013 1877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:43.139040 1877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "5bbe3af094b64fbab9365f3624421c7c"
format_stamp: "Formatted at 2026-08-09 03:32:43 on dist-test-slave-gkw7"
server_key: "f315d364d283c4cb59083a95c33fd25a"
server_key_iv: "e3da44888faeba968d26b70a02a57e42"
server_key_version: "encryptionkey@0"
I20260809 03:32:43.139420 1877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "5bbe3af094b64fbab9365f3624421c7c"
format_stamp: "Formatted at 2026-08-09 03:32:43 on dist-test-slave-gkw7"
server_key: "f315d364d283c4cb59083a95c33fd25a"
server_key_iv: "e3da44888faeba968d26b70a02a57e42"
server_key_version: "encryptionkey@0"
I20260809 03:32:43.140805 1877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:32:43.141641 1900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:43.141840 1877 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:32:43.141943 1877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "5bbe3af094b64fbab9365f3624421c7c"
format_stamp: "Formatted at 2026-08-09 03:32:43 on dist-test-slave-gkw7"
server_key: "f315d364d283c4cb59083a95c33fd25a"
server_key_iv: "e3da44888faeba968d26b70a02a57e42"
server_key_version: "encryptionkey@0"
I20260809 03:32:43.142024 1877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:43.160297 1877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:43.162355 1877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:43.162515 1877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:43.162760 1877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:43.163079 1877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:43.163142 1877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:43.163206 1877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:43.163239 1877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:43.169375 1877 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:38971
I20260809 03:32:43.169415 2071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:38971 every 8 connection(s)
I20260809 03:32:43.169727 1877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:32:43.176329 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 1877
I20260809 03:32:43.176457 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:32:43.176661 27810 external_mini_cluster.cc:1428] Setting key d93ff94ef8a9eee1732210bfe915f870
Aug 09 03:32:43 dist-test-slave-gkw7 krb5kdc[1539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246363, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:43.181020 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.170440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:45303 (local address 127.27.40.190:38693)
0809 03:32:43.170517 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:43.170519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:43.170726 (+ 207us) server_negotiation.cc:407] Connection header received
0809 03:32:43.171278 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:43.171279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:43.171293 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:43.171358 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:43.172599 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.173381 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.175000 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.175387 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.175451 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.177646 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:43.177648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:43.177648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:43.177657 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:43.179380 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:43.179954 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:43.179955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:43.179956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:43.180012 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:43.180339 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:43.180340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:43.180340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:43.180462 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:43.180530 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:43.180874 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 03:32:43.180954 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:32:43.181360 2076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.170451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:38693 (local address 127.27.40.130:45303)
0809 03:32:43.170600 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:43.170637 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:43.171165 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:43.171456 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:43.171460 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:43.171824 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:43.172477 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:43.172484 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.173501 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.173501 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:43.174850 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:43.174853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.175460 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.175461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:43.175582 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.175983 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:43.175990 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:43.177565 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:43.179505 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:43.179514 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:43.179516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:43.179862 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:43.180092 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:43.180093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:43.180094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:43.180248 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:43.180539 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:43.180546 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:43.180726 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 03:32:43.180854 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 03:32:43.181039 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260809 03:32:43.181941 2074 heartbeater.cc:347] Connected to a master server at 127.27.40.190:38693
I20260809 03:32:43.182039 2074 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:43.182235 2074 heartbeater.cc:511] Master 127.27.40.190:38693 requested a full tablet report, sending...
I20260809 03:32:43.182686 1597 ts_manager.cc:194] Registered new tserver with Master: 5bbe3af094b64fbab9365f3624421c7c (127.27.40.130:38971)
I20260809 03:32:43.183163 1597 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:45303
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:32:43.205150 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38693
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:43.291299 2079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:43.291491 2079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:43.291515 2079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:43.292572 2079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:43.292603 2079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:43.292629 2079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:32:43.294198 2079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:43.294370 2079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:43.294534 2079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:43.299510 2085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:43.299515 2093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:43.299522 2088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:43.299548 2079 server_base.cc:1034] running on GCE node
I20260809 03:32:43.299904 2079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:43.300160 2079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:43.301312 2079 hybrid_clock.cc:648] HybridClock initialized: now 1786246363301300 us; error 27 us; skew 500 ppm
Aug 09 03:32:43 dist-test-slave-gkw7 krb5kdc[1539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246363, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:43.303802 2079 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:32:43.304160 2079 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:43.304209 2079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:43.304304 2079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:43.305272 2079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "ccd48dba4cd441fa8634259a1444b370"
format_stamp: "Formatted at 2026-08-09 03:32:43 on dist-test-slave-gkw7"
server_key: "c32884fecca1a70e97c70783470d6c65"
server_key_iv: "0f5ffb3ed5e711362beea87e996fd709"
server_key_version: "encryptionkey@0"
I20260809 03:32:43.305671 2079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "ccd48dba4cd441fa8634259a1444b370"
format_stamp: "Formatted at 2026-08-09 03:32:43 on dist-test-slave-gkw7"
server_key: "c32884fecca1a70e97c70783470d6c65"
server_key_iv: "0f5ffb3ed5e711362beea87e996fd709"
server_key_version: "encryptionkey@0"
I20260809 03:32:43.307169 2079 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:32:43.307950 2099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:43.308115 2079 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:32:43.308233 2079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "ccd48dba4cd441fa8634259a1444b370"
format_stamp: "Formatted at 2026-08-09 03:32:43 on dist-test-slave-gkw7"
server_key: "c32884fecca1a70e97c70783470d6c65"
server_key_iv: "0f5ffb3ed5e711362beea87e996fd709"
server_key_version: "encryptionkey@0"
I20260809 03:32:43.308331 2079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:43.321409 2079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:43.323180 2079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:43.323345 2079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:43.323571 2079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:43.323876 2079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:43.323931 2079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:43.323971 2079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:43.323999 2079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:43.330483 2079 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:34745
I20260809 03:32:43.330524 2270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:34745 every 8 connection(s)
I20260809 03:32:43.330806 2079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
Aug 09 03:32:43 dist-test-slave-gkw7 krb5kdc[1539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246363, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:43.339741 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.331541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:48487 (local address 127.27.40.190:38693)
0809 03:32:43.331596 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:43.331598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:43.331805 (+ 207us) server_negotiation.cc:407] Connection header received
0809 03:32:43.332377 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:43.332378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:43.332391 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:43.332450 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:43.333379 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.333962 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.334957 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.335158 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.335235 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.337382 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:43.337383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:43.337383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:43.337390 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:43.338612 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:43.338976 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:43.338976 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:43.338977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:43.339028 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:43.339233 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:43.339233 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:43.339233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:43.339331 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:43.339383 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:43.339609 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 03:32:43.339678 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 03:32:43.339886 2275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.331512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:38693 (local address 127.27.40.131:48487)
0809 03:32:43.331692 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:43.331725 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:43.332270 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:43.332517 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:43.332522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:43.332777 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:43.333266 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:43.333270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.334057 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.334058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:43.334858 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:43.334860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.335242 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.335243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:43.335368 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.335870 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:43.335878 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:43.337299 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:43.338693 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:43.338699 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:43.338700 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:43.338912 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:43.339086 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:43.339086 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:43.339087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:43.339180 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:43.339387 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:43.339392 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:43.339502 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 03:32:43.339581 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 03:32:43.339697 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260809 03:32:43.340400 2271 heartbeater.cc:347] Connected to a master server at 127.27.40.190:38693
I20260809 03:32:43.340513 2271 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:43.340637 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 2079
I20260809 03:32:43.340675 2271 heartbeater.cc:511] Master 127.27.40.190:38693 requested a full tablet report, sending...
I20260809 03:32:43.340739 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:32:43.340973 27810 external_mini_cluster.cc:1428] Setting key e902aed4e68b8d24bded2da96d27464f
I20260809 03:32:43.341099 1597 ts_manager.cc:194] Registered new tserver with Master: ccd48dba4cd441fa8634259a1444b370 (127.27.40.131:34745)
I20260809 03:32:43.341612 1597 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:48487
I20260809 03:32:43.341981 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:32:43.348914 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.343242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40546 (local address 127.27.40.190:38693)
0809 03:32:43.343281 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:43.343283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:43.343526 (+ 243us) server_negotiation.cc:407] Connection header received
0809 03:32:43.343572 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:43.343573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:43.343581 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:43.343625 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:43.344229 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.344669 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.345421 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.345651 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.345702 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.346879 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:43.346880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:43.346880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:43.346886 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:43.347994 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:43.348293 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:43.348294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:43.348294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:43.348343 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:43.348476 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:43.348476 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:43.348476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:43.348598 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:43.348679 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:43.348779 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 03:32:43.348834 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 03:32:43.350620 1596 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40546:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:32:43.352274 1596 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:43.359047 2287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.354535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:36749 (local address 127.0.0.1:51288)
0809 03:32:43.354770 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:43.354810 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:43.354879 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:43.355499 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:43.355500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:43.355502 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:43.355558 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:43.355559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.356843 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.356844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:43.357945 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:43.357948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.358713 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.358714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:43.358800 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.358814 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 03:32:43.358883 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 03:32:43.358936 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":93,"threads_started":1}
I20260809 03:32:43.359349 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.354821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48440 (local address 127.27.40.130:38971)
0809 03:32:43.354993 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:43.354998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:43.355008 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:32:43.355047 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:43.355049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:43.355063 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:43.355167 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:43.355341 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.356556 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.358080 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.358798 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.358871 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.358919 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:43.359082 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 03:32:43.359267 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":69,"threads_started":1}
I20260809 03:32:43.359373 2290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.354663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51288 (local address 127.27.40.129:36749)
0809 03:32:43.355332 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:43.355336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:43.355345 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:43.355387 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:43.355389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:43.355405 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:43.355491 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:43.355620 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.356703 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.358036 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.358619 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.358712 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.358783 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:43.359208 (+ 425us) server_negotiation.cc:299] Negotiation successful
0809 03:32:43.359252 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":78,"threads_started":1}
I20260809 03:32:43.360332 2286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.354590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:34745 (local address 127.0.0.1:57784)
0809 03:32:43.354831 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:43.354841 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:43.354881 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:43.357054 (+ 2173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:43.357055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:43.357058 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:43.357102 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:43.357104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.358152 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.358152 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:43.359211 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:43.359213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.360174 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.360175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:43.360221 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.360225 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:43.360257 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 03:32:43.360286 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":1,"thread_start_us":74,"threads_started":1}
I20260809 03:32:43.360388 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.354748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57784 (local address 127.27.40.131:34745)
0809 03:32:43.356907 (+ 2159us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:43.356910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:43.356918 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:32:43.356946 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:43.356947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:43.356958 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:43.357055 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:43.357159 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.358036 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.359280 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.360074 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.360188 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.360243 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:43.360279 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 03:32:43.360315 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2120,"thread_start_us":87,"threads_started":1}
I20260809 03:32:43.359180 2288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.354524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:38971 (local address 127.0.0.1:48440)
0809 03:32:43.354899 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:43.354911 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:43.354968 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:43.355176 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:43.355179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:43.355181 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:43.355248 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:43.355250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.356652 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.356652 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:43.358010 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:43.358012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.358870 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.358871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:43.358994 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.358998 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:43.359055 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 03:32:43.359100 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":3,"thread_start_us":61,"threads_started":1}
I20260809 03:32:43.360666 1772 tablet_service.cc:1467] Processing CreateTablet for tablet bd5af52e2163497ea21388919031ad5b (DEFAULT_TABLE table=test-table [id=3ec21e25d0034ef281f0077bbb24c7a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:43.361084 1772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd5af52e2163497ea21388919031ad5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:43.361142 2172 tablet_service.cc:1467] Processing CreateTablet for tablet bd5af52e2163497ea21388919031ad5b (DEFAULT_TABLE table=test-table [id=3ec21e25d0034ef281f0077bbb24c7a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:43.361478 2172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd5af52e2163497ea21388919031ad5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:43.361393 1969 tablet_service.cc:1467] Processing CreateTablet for tablet bd5af52e2163497ea21388919031ad5b (DEFAULT_TABLE table=test-table [id=3ec21e25d0034ef281f0077bbb24c7a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:43.361792 1969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd5af52e2163497ea21388919031ad5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:43.364650 2294 tablet_bootstrap.cc:492] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370: Bootstrap starting.
I20260809 03:32:43.365480 2294 tablet_bootstrap.cc:654] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:43.366667 2294 log.cc:824] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:43.366755 2297 tablet_bootstrap.cc:492] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c: Bootstrap starting.
I20260809 03:32:43.367066 2295 tablet_bootstrap.cc:492] T bd5af52e2163497ea21388919031ad5b P 16097ae5301b4f7eb8cfc5efc0bdfec3: Bootstrap starting.
I20260809 03:32:43.367594 2294 tablet_bootstrap.cc:492] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370: No bootstrap required, opened a new log
I20260809 03:32:43.367679 2294 ts_tablet_manager.cc:1397] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:32:43.368304 2297 tablet_bootstrap.cc:654] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:43.368404 2295 tablet_bootstrap.cc:654] T bd5af52e2163497ea21388919031ad5b P 16097ae5301b4f7eb8cfc5efc0bdfec3: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:43.369678 2295 log.cc:824] T bd5af52e2163497ea21388919031ad5b P 16097ae5301b4f7eb8cfc5efc0bdfec3: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:43.369673 2297 log.cc:824] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:43.370483 2294 raft_consensus.cc:348] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbe3af094b64fbab9365f3624421c7c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 38971 } } peers { permanent_uuid: "16097ae5301b4f7eb8cfc5efc0bdfec3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36749 } } peers { permanent_uuid: "ccd48dba4cd441fa8634259a1444b370" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34745 } }
I20260809 03:32:43.370654 2294 raft_consensus.cc:374] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:43.370699 2294 raft_consensus.cc:729] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccd48dba4cd441fa8634259a1444b370, State: Initialized, Role: FOLLOWER
I20260809 03:32:43.370731 2295 tablet_bootstrap.cc:492] T bd5af52e2163497ea21388919031ad5b P 16097ae5301b4f7eb8cfc5efc0bdfec3: No bootstrap required, opened a new log
I20260809 03:32:43.370808 2295 ts_tablet_manager.cc:1397] T bd5af52e2163497ea21388919031ad5b P 16097ae5301b4f7eb8cfc5efc0bdfec3: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 03:32:43.370810 2294 consensus_queue.cc:260] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbe3af094b64fbab9365f3624421c7c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 38971 } } peers { permanent_uuid: "16097ae5301b4f7eb8cfc5efc0bdfec3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36749 } } peers { permanent_uuid: "ccd48dba4cd441fa8634259a1444b370" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34745 } }
I20260809 03:32:43.371258 2271 heartbeater.cc:503] Master 127.27.40.190:38693 was elected leader, sending a full tablet report...
I20260809 03:32:43.371486 2294 ts_tablet_manager.cc:1428] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 03:32:43.372330 2297 tablet_bootstrap.cc:492] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c: No bootstrap required, opened a new log
I20260809 03:32:43.372423 2297 ts_tablet_manager.cc:1397] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260809 03:32:43.373505 2295 raft_consensus.cc:348] T bd5af52e2163497ea21388919031ad5b P 16097ae5301b4f7eb8cfc5efc0bdfec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbe3af094b64fbab9365f3624421c7c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 38971 } } peers { permanent_uuid: "16097ae5301b4f7eb8cfc5efc0bdfec3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36749 } } peers { permanent_uuid: "ccd48dba4cd441fa8634259a1444b370" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34745 } }
I20260809 03:32:43.373696 2295 raft_consensus.cc:374] T bd5af52e2163497ea21388919031ad5b P 16097ae5301b4f7eb8cfc5efc0bdfec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:43.373734 2295 raft_consensus.cc:729] T bd5af52e2163497ea21388919031ad5b P 16097ae5301b4f7eb8cfc5efc0bdfec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16097ae5301b4f7eb8cfc5efc0bdfec3, State: Initialized, Role: FOLLOWER
I20260809 03:32:43.373885 2295 consensus_queue.cc:260] T bd5af52e2163497ea21388919031ad5b P 16097ae5301b4f7eb8cfc5efc0bdfec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbe3af094b64fbab9365f3624421c7c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 38971 } } peers { permanent_uuid: "16097ae5301b4f7eb8cfc5efc0bdfec3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36749 } } peers { permanent_uuid: "ccd48dba4cd441fa8634259a1444b370" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34745 } }
I20260809 03:32:43.374243 1872 heartbeater.cc:503] Master 127.27.40.190:38693 was elected leader, sending a full tablet report...
I20260809 03:32:43.374189 2295 ts_tablet_manager.cc:1428] T bd5af52e2163497ea21388919031ad5b P 16097ae5301b4f7eb8cfc5efc0bdfec3: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260809 03:32:43.375459 2297 raft_consensus.cc:348] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbe3af094b64fbab9365f3624421c7c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 38971 } } peers { permanent_uuid: "16097ae5301b4f7eb8cfc5efc0bdfec3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36749 } } peers { permanent_uuid: "ccd48dba4cd441fa8634259a1444b370" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34745 } }
I20260809 03:32:43.375617 2297 raft_consensus.cc:374] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:43.375676 2297 raft_consensus.cc:729] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bbe3af094b64fbab9365f3624421c7c, State: Initialized, Role: FOLLOWER
I20260809 03:32:43.375789 2297 consensus_queue.cc:260] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbe3af094b64fbab9365f3624421c7c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 38971 } } peers { permanent_uuid: "16097ae5301b4f7eb8cfc5efc0bdfec3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36749 } } peers { permanent_uuid: "ccd48dba4cd441fa8634259a1444b370" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34745 } }
I20260809 03:32:43.376055 2297 ts_tablet_manager.cc:1428] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260809 03:32:43.376231 2074 heartbeater.cc:503] Master 127.27.40.190:38693 was elected leader, sending a full tablet report...
W20260809 03:32:43.420295 2075 tablet.cc:2369] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:32:43.505306 1873 tablet.cc:2369] T bd5af52e2163497ea21388919031ad5b P 16097ae5301b4f7eb8cfc5efc0bdfec3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:32:43.581393 2274 tablet.cc:2369] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:32:43.584475 2301 raft_consensus.cc:482] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:43.584578 2301 raft_consensus.cc:504] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbe3af094b64fbab9365f3624421c7c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 38971 } } peers { permanent_uuid: "16097ae5301b4f7eb8cfc5efc0bdfec3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36749 } } peers { permanent_uuid: "ccd48dba4cd441fa8634259a1444b370" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34745 } }
I20260809 03:32:43.585088 2301 leader_election.cc:288] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bbe3af094b64fbab9365f3624421c7c (127.27.40.130:38971), 16097ae5301b4f7eb8cfc5efc0bdfec3 (127.27.40.129:36749)
I20260809 03:32:43.589381 2275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.585298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:36749 (local address 127.27.40.131:45059)
0809 03:32:43.585367 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:43.585379 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:43.585424 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:43.585636 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:43.585637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:43.585639 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:43.585808 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:43.585809 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.586804 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.586805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:43.588231 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:43.588234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.589072 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.589074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:43.589228 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.589232 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:43.589285 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 03:32:43.589314 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 03:32:43.589395 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.585370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:58319 (local address 127.27.40.130:38971)
0809 03:32:43.585425 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:43.585428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:43.585676 (+ 248us) server_negotiation.cc:407] Connection header received
0809 03:32:43.585716 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:43.585717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:43.585725 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:43.585771 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:43.586045 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.586790 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.588337 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.588993 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.589063 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.589073 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:43.589257 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 03:32:43.589325 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 03:32:43.589416 2290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.585355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:45059 (local address 127.27.40.129:36749)
0809 03:32:43.585407 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:43.585409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:43.585418 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:43.585510 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:43.585512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:43.585521 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:43.585628 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:43.585908 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.586716 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.588336 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.588960 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.589051 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.589073 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:43.589310 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 03:32:43.589351 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 03:32:43.589388 2306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.585323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:38971 (local address 127.27.40.131:58319)
0809 03:32:43.585588 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:43.585598 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:43.585663 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:43.585795 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:43.585796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:43.585797 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:43.585963 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:43.585965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.586867 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.586868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:43.588231 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:43.588234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.589073 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.589074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:43.589154 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.589159 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:32:43.589241 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 03:32:43.589294 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":3,"thread_start_us":117,"threads_started":1}
I20260809 03:32:43.589763 1799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5af52e2163497ea21388919031ad5b" candidate_uuid: "ccd48dba4cd441fa8634259a1444b370" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16097ae5301b4f7eb8cfc5efc0bdfec3" is_pre_election: true
I20260809 03:32:43.589833 2001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5af52e2163497ea21388919031ad5b" candidate_uuid: "ccd48dba4cd441fa8634259a1444b370" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbe3af094b64fbab9365f3624421c7c" is_pre_election: true
I20260809 03:32:43.589941 1799 raft_consensus.cc:2454] T bd5af52e2163497ea21388919031ad5b P 16097ae5301b4f7eb8cfc5efc0bdfec3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccd48dba4cd441fa8634259a1444b370 in term 0.
I20260809 03:32:43.589963 2001 raft_consensus.cc:2454] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccd48dba4cd441fa8634259a1444b370 in term 0.
I20260809 03:32:43.590204 2102 leader_election.cc:302] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16097ae5301b4f7eb8cfc5efc0bdfec3, ccd48dba4cd441fa8634259a1444b370; no voters:
I20260809 03:32:43.590395 2301 raft_consensus.cc:2781] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:43.590498 2301 raft_consensus.cc:482] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:43.590531 2301 raft_consensus.cc:3037] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:43.591324 2301 raft_consensus.cc:504] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbe3af094b64fbab9365f3624421c7c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 38971 } } peers { permanent_uuid: "16097ae5301b4f7eb8cfc5efc0bdfec3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36749 } } peers { permanent_uuid: "ccd48dba4cd441fa8634259a1444b370" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34745 } }
I20260809 03:32:43.591485 2301 leader_election.cc:288] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [CANDIDATE]: Term 1 election: Requested vote from peers 5bbe3af094b64fbab9365f3624421c7c (127.27.40.130:38971), 16097ae5301b4f7eb8cfc5efc0bdfec3 (127.27.40.129:36749)
I20260809 03:32:43.591631 2001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5af52e2163497ea21388919031ad5b" candidate_uuid: "ccd48dba4cd441fa8634259a1444b370" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbe3af094b64fbab9365f3624421c7c"
I20260809 03:32:43.591692 1799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5af52e2163497ea21388919031ad5b" candidate_uuid: "ccd48dba4cd441fa8634259a1444b370" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16097ae5301b4f7eb8cfc5efc0bdfec3"
I20260809 03:32:43.591735 2001 raft_consensus.cc:3037] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:43.591782 1799 raft_consensus.cc:3037] T bd5af52e2163497ea21388919031ad5b P 16097ae5301b4f7eb8cfc5efc0bdfec3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:43.592595 1799 raft_consensus.cc:2454] T bd5af52e2163497ea21388919031ad5b P 16097ae5301b4f7eb8cfc5efc0bdfec3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccd48dba4cd441fa8634259a1444b370 in term 1.
I20260809 03:32:43.592700 2001 raft_consensus.cc:2454] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccd48dba4cd441fa8634259a1444b370 in term 1.
I20260809 03:32:43.592779 2102 leader_election.cc:302] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16097ae5301b4f7eb8cfc5efc0bdfec3, ccd48dba4cd441fa8634259a1444b370; no voters:
I20260809 03:32:43.592938 2301 raft_consensus.cc:2781] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:43.593111 2301 raft_consensus.cc:686] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [term 1 LEADER]: Becoming Leader. State: Replica: ccd48dba4cd441fa8634259a1444b370, State: Running, Role: LEADER
I20260809 03:32:43.593226 2301 consensus_queue.cc:237] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbe3af094b64fbab9365f3624421c7c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 38971 } } peers { permanent_uuid: "16097ae5301b4f7eb8cfc5efc0bdfec3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36749 } } peers { permanent_uuid: "ccd48dba4cd441fa8634259a1444b370" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34745 } }
I20260809 03:32:43.594169 1596 catalog_manager.cc:5495] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccd48dba4cd441fa8634259a1444b370 (127.27.40.131). New cstate: current_term: 1 leader_uuid: "ccd48dba4cd441fa8634259a1444b370" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbe3af094b64fbab9365f3624421c7c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 38971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16097ae5301b4f7eb8cfc5efc0bdfec3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccd48dba4cd441fa8634259a1444b370" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34745 } health_report { overall_health: HEALTHY } } }
I20260809 03:32:43.641341 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.638936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57790 (local address 127.27.40.131:34745)
0809 03:32:43.639009 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:43.639011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:43.639021 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:32:43.639076 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:43.639077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:43.639095 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:43.639173 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:43.639347 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.639829 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.640568 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:43.640752 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:43.640802 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:43.640977 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:43.641077 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:43.641138 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:43.641184 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 03:32:43.641201 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:32:43.643754 2001 raft_consensus.cc:1261] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c [term 1 FOLLOWER]: Refusing update from remote peer ccd48dba4cd441fa8634259a1444b370: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:43.643762 1799 raft_consensus.cc:1261] T bd5af52e2163497ea21388919031ad5b P 16097ae5301b4f7eb8cfc5efc0bdfec3 [term 1 FOLLOWER]: Refusing update from remote peer ccd48dba4cd441fa8634259a1444b370: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:43.644117 2307 consensus_queue.cc:1035] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16097ae5301b4f7eb8cfc5efc0bdfec3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:43.644248 2301 consensus_queue.cc:1035] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bbe3af094b64fbab9365f3624421c7c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 38971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:43.646894 2314 mvcc.cc:205] Tried to move back new op lower bound from 7316465105481732096 to 7316465105278525440. Current Snapshot: MvccSnapshot[applied={T|T < 7316465105481732096}]
I20260809 03:32:43.647051 2315 mvcc.cc:205] Tried to move back new op lower bound from 7316465105481732096 to 7316465105278525440. Current Snapshot: MvccSnapshot[applied={T|T < 7316465105481732096}]
I20260809 03:32:43.647547 2313 mvcc.cc:205] Tried to move back new op lower bound from 7316465105481732096 to 7316465105278525440. Current Snapshot: MvccSnapshot[applied={T|T < 7316465105481732096 or (T in {7316465105481732096})}]
I20260809 03:32:43.648974 1596 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:43.649070 1596 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:43.649868 1596 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b9a88f289e274d3ca77026162f245c1b: Sending DeleteTablet for 3 replicas of tablet bd5af52e2163497ea21388919031ad5b
I20260809 03:32:43.650209 1969 tablet_service.cc:1514] Processing DeleteTablet for tablet bd5af52e2163497ea21388919031ad5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:43 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:48440
I20260809 03:32:43.650285 1772 tablet_service.cc:1514] Processing DeleteTablet for tablet bd5af52e2163497ea21388919031ad5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:43 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:51288
I20260809 03:32:43.650420 2172 tablet_service.cc:1514] Processing DeleteTablet for tablet bd5af52e2163497ea21388919031ad5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:43 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:57784
I20260809 03:32:43.650552 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 1690
I20260809 03:32:43.661069 2329 tablet_replica.cc:330] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370: stopping tablet replica
I20260809 03:32:43.661206 2329 raft_consensus.cc:2229] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:32:43.661320 2329 raft_consensus.cc:2258] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:43.666935 2327 tablet_replica.cc:330] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c: stopping tablet replica
I20260809 03:32:43.667074 2327 raft_consensus.cc:2229] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:43.667201 2327 raft_consensus.cc:2258] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:43.667505 2329 ts_tablet_manager.cc:1905] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:43.667703 2327 ts_tablet_manager.cc:1905] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:43.669409 2329 ts_tablet_manager.cc:1918] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:43.669488 2329 log.cc:1196] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/bd5af52e2163497ea21388919031ad5b
I20260809 03:32:43.669798 2329 ts_tablet_manager.cc:1939] T bd5af52e2163497ea21388919031ad5b P ccd48dba4cd441fa8634259a1444b370: Deleting consensus metadata
I20260809 03:32:43.669807 2327 ts_tablet_manager.cc:1918] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:43.669876 2327 log.cc:1196] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/bd5af52e2163497ea21388919031ad5b
I20260809 03:32:43.670135 2327 ts_tablet_manager.cc:1939] T bd5af52e2163497ea21388919031ad5b P 5bbe3af094b64fbab9365f3624421c7c: Deleting consensus metadata
I20260809 03:32:43.670253 1576 catalog_manager.cc:4841] TS ccd48dba4cd441fa8634259a1444b370 (127.27.40.131:34745): tablet bd5af52e2163497ea21388919031ad5b (table test-table [id=3ec21e25d0034ef281f0077bbb24c7a7]) successfully deleted
I20260809 03:32:43.670567 1578 catalog_manager.cc:4841] TS 5bbe3af094b64fbab9365f3624421c7c (127.27.40.130:38971): tablet bd5af52e2163497ea21388919031ad5b (table test-table [id=3ec21e25d0034ef281f0077bbb24c7a7]) successfully deleted
W20260809 03:32:43.672029 1577 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.40.129:36749 (error 108)
I20260809 03:32:43.673734 2288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.672476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:36749 (local address 127.0.0.1:51296)
0809 03:32:43.673638 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:43.673698 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:36749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1125}
I20260809 03:32:43.673830 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 1877
W20260809 03:32:43.673856 1577 catalog_manager.cc:4584] TS 16097ae5301b4f7eb8cfc5efc0bdfec3 (127.27.40.129:36749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd5af52e2163497ea21388919031ad5b: Network error: Client connection negotiation failed: client connection to 127.27.40.129:36749: connect: Connection refused (error 111)
I20260809 03:32:43.697364 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 2079
I20260809 03:32:43.706818 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 1553
2026-08-09T03:32:43Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2465 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T03:32:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:43Z Disabled control of system clock
I20260809 03:32:43.740720 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:37565
--webserver_interface=127.27.40.190
--webserver_port=0
--builtin_ntp_servers=127.27.40.148:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:37565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 03:32:43.824994 2334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:43.825233 2334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:43.825269 2334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:43.826440 2334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:43.826510 2334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:43.826532 2334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:43.826555 2334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:43.826584 2334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:32:43.827903 2334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:37565
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:37565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:43.828183 2334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:43.828474 2334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:43.833381 2344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:43.833379 2342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:43.833500 2334 server_base.cc:1034] running on GCE node
W20260809 03:32:43.833444 2339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:43.833932 2334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:43.834240 2334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:43.835386 2334 hybrid_clock.cc:648] HybridClock initialized: now 1786246363835373 us; error 29 us; skew 500 ppm
I20260809 03:32:43.836594 2334 webserver.cc:460] Webserver started at http://127.27.40.190:36543/ using document root <none> and password file <none>
I20260809 03:32:43.836833 2334 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:43.836877 2334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:43.837026 2334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:43.838166 2334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "470eb8ef1094461ebb42fc79b9a0a5c3"
format_stamp: "Formatted at 2026-08-09 03:32:43 on dist-test-slave-gkw7"
server_key: "9c4b7ec6b14a8ccf50a6581a565b4bc1"
server_key_iv: "e5372e6634189fe301ce53ce58786da6"
server_key_version: "encryptionkey@0"
I20260809 03:32:43.838517 2334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "470eb8ef1094461ebb42fc79b9a0a5c3"
format_stamp: "Formatted at 2026-08-09 03:32:43 on dist-test-slave-gkw7"
server_key: "9c4b7ec6b14a8ccf50a6581a565b4bc1"
server_key_iv: "e5372e6634189fe301ce53ce58786da6"
server_key_version: "encryptionkey@0"
I20260809 03:32:43.839785 2334 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:32:43.840513 2359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:43.840691 2334 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:43.840780 2334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "470eb8ef1094461ebb42fc79b9a0a5c3"
format_stamp: "Formatted at 2026-08-09 03:32:43 on dist-test-slave-gkw7"
server_key: "9c4b7ec6b14a8ccf50a6581a565b4bc1"
server_key_iv: "e5372e6634189fe301ce53ce58786da6"
server_key_version: "encryptionkey@0"
I20260809 03:32:43.840853 2334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:43.844152 2334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:43.844426 2334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:43.844563 2334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:43.848588 2334 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:37565
I20260809 03:32:43.848616 2435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:37565 every 8 connection(s)
I20260809 03:32:43.848929 2334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:43.849606 2436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:43.852242 2436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3: Bootstrap starting.
I20260809 03:32:43.852860 2436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:43.853169 2436 log.cc:824] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:43.853935 2436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3: No bootstrap required, opened a new log
I20260809 03:32:43.855333 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 2334
I20260809 03:32:43.855432 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:43.855623 27810 external_mini_cluster.cc:1428] Setting key b66154ec9b60a6e57a8c72307c7161eb
I20260809 03:32:43.855839 2436 raft_consensus.cc:348] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470eb8ef1094461ebb42fc79b9a0a5c3" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37565 } }
I20260809 03:32:43.855998 2436 raft_consensus.cc:374] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:43.856034 2436 raft_consensus.cc:729] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470eb8ef1094461ebb42fc79b9a0a5c3, State: Initialized, Role: FOLLOWER
I20260809 03:32:43.856166 2436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470eb8ef1094461ebb42fc79b9a0a5c3" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37565 } }
I20260809 03:32:43.856281 2436 raft_consensus.cc:388] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:43.856328 2436 raft_consensus.cc:482] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:43.856379 2436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:43.857008 2436 raft_consensus.cc:504] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470eb8ef1094461ebb42fc79b9a0a5c3" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37565 } }
I20260809 03:32:43.857146 2436 leader_election.cc:302] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 470eb8ef1094461ebb42fc79b9a0a5c3; no voters:
I20260809 03:32:43.857318 2436 leader_election.cc:288] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:43.857400 2445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:43.857566 2445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [term 1 LEADER]: Becoming Leader. State: Replica: 470eb8ef1094461ebb42fc79b9a0a5c3, State: Running, Role: LEADER
I20260809 03:32:43.857611 2436 sys_catalog.cc:564] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:43.857671 2445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470eb8ef1094461ebb42fc79b9a0a5c3" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37565 } }
I20260809 03:32:43.858233 2447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 470eb8ef1094461ebb42fc79b9a0a5c3. Latest consensus state: current_term: 1 leader_uuid: "470eb8ef1094461ebb42fc79b9a0a5c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470eb8ef1094461ebb42fc79b9a0a5c3" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37565 } } }
I20260809 03:32:43.858335 2447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:43.858506 2446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "470eb8ef1094461ebb42fc79b9a0a5c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470eb8ef1094461ebb42fc79b9a0a5c3" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37565 } } }
I20260809 03:32:43.858546 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.856113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60952 (local address 127.27.40.190:37565)
0809 03:32:43.856378 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:43.856387 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:43.856486 (+ 99us) server_negotiation.cc:407] Connection header received
0809 03:32:43.856629 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:43.856634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:43.856666 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:43.856771 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:43.857567 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:43.857571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:43.857571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:32:43.857576 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:43.857697 (+ 121us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:32:43.857717 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:43.858134 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:43.858190 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 03:32:43.858250 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":107,"threads_started":1}
I20260809 03:32:43.858567 2446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3 [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:43.858549 2460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:43.859225 2460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:43.860724 2460 catalog_manager.cc:1347] Generated new cluster ID: 206bb6ae46354c1398c6693f303cf6db
I20260809 03:32:43.860775 2460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:32:43.871259 2460 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:43.871412 2460 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:43.877727 2460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3: Generated new TSK 0
I20260809 03:32:43.877902 2460 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:32:43.881642 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37565
--builtin_ntp_servers=127.27.40.148:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 03:32:43.964027 2474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:43.964224 2474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:43.964246 2474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:43.965409 2474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:43.965449 2474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:43.965476 2474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:32:43.966841 2474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:43.967084 2474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:43.967401 2474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:43.973569 2481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:43.973701 2480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:43.973599 2474 server_base.cc:1034] running on GCE node
W20260809 03:32:43.973619 2486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:43.974086 2474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:43.974359 2474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:43.975525 2474 hybrid_clock.cc:648] HybridClock initialized: now 1786246363975502 us; error 40 us; skew 500 ppm
I20260809 03:32:43.976840 2474 webserver.cc:460] Webserver started at http://127.27.40.129:46251/ using document root <none> and password file <none>
I20260809 03:32:43.977121 2474 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:43.977218 2474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:43.977421 2474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:43.978561 2474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "35b34fbea0e64ffd8ec3f6fd6100fd67"
format_stamp: "Formatted at 2026-08-09 03:32:43 on dist-test-slave-gkw7"
server_key: "8e64d2b2c766a99e9393808cec989132"
server_key_iv: "736d8d18398f283d634367023f98f1eb"
server_key_version: "encryptionkey@0"
I20260809 03:32:43.978938 2474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "35b34fbea0e64ffd8ec3f6fd6100fd67"
format_stamp: "Formatted at 2026-08-09 03:32:43 on dist-test-slave-gkw7"
server_key: "8e64d2b2c766a99e9393808cec989132"
server_key_iv: "736d8d18398f283d634367023f98f1eb"
server_key_version: "encryptionkey@0"
I20260809 03:32:43.980315 2474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 03:32:43.981076 2496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:43.981254 2474 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:32:43.981357 2474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "35b34fbea0e64ffd8ec3f6fd6100fd67"
format_stamp: "Formatted at 2026-08-09 03:32:43 on dist-test-slave-gkw7"
server_key: "8e64d2b2c766a99e9393808cec989132"
server_key_iv: "736d8d18398f283d634367023f98f1eb"
server_key_version: "encryptionkey@0"
I20260809 03:32:43.981438 2474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:43.984841 2474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:43.985126 2474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:43.985284 2474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:43.985504 2474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:43.985843 2474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:43.985899 2474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:43.985939 2474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:43.985965 2474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:43.991896 2474 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:44663
I20260809 03:32:43.991974 2664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:44663 every 8 connection(s)
I20260809 03:32:43.992308 2474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:32:43.995298 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.993075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:58287 (local address 127.27.40.190:37565)
0809 03:32:43.993184 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:43.993186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:43.993366 (+ 180us) server_negotiation.cc:407] Connection header received
0809 03:32:43.993916 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:43.993917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:43.993924 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:43.993990 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:43.995108 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:43.995109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:43.995109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:32:43.995114 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:43.995131 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:32:43.995135 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:43.995185 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:43.995232 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 03:32:43.995244 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 03:32:43.995512 2667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:43.993036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37565 (local address 127.27.40.129:58287)
0809 03:32:43.993244 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:43.993277 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:43.993789 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:43.994044 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:43.994047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:43.994055 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:43.994943 (+ 888us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:32:43.994944 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:43.994971 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:32:43.994997 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:43.995182 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:43.995185 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:43.995187 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 03:32:43.995237 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 03:32:43.995283 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":110,"threads_started":1}
I20260809 03:32:43.996019 2665 heartbeater.cc:347] Connected to a master server at 127.27.40.190:37565
I20260809 03:32:43.996155 2665 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:43.996325 2665 heartbeater.cc:511] Master 127.27.40.190:37565 requested a full tablet report, sending...
I20260809 03:32:43.996829 2383 ts_manager.cc:194] Registered new tserver with Master: 35b34fbea0e64ffd8ec3f6fd6100fd67 (127.27.40.129:44663)
I20260809 03:32:43.997771 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 2474
I20260809 03:32:43.997849 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:32:43.998039 27810 external_mini_cluster.cc:1428] Setting key a44ef898ed4c83b4b9b9aaa6c6b2bb18
I20260809 03:32:43.998883 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37565
--builtin_ntp_servers=127.27.40.148:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 03:32:44.081676 2670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:44.081898 2670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:44.081921 2670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:44.083048 2670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:44.083083 2670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:44.083108 2670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:32:44.084558 2670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:44.084874 2670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:44.085232 2670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:44.090341 2675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:44.090391 2681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:44.090399 2677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:44.090658 2670 server_base.cc:1034] running on GCE node
I20260809 03:32:44.090866 2670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:44.091061 2670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:44.092242 2670 hybrid_clock.cc:648] HybridClock initialized: now 1786246364092222 us; error 27 us; skew 500 ppm
I20260809 03:32:44.093613 2670 webserver.cc:460] Webserver started at http://127.27.40.130:39511/ using document root <none> and password file <none>
I20260809 03:32:44.093837 2670 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:44.093879 2670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:44.094022 2670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:44.095089 2670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b"
format_stamp: "Formatted at 2026-08-09 03:32:44 on dist-test-slave-gkw7"
server_key: "927ad28b381a344ee60f50140255af42"
server_key_iv: "60f0a24f9c41e635142875af3594a689"
server_key_version: "encryptionkey@0"
I20260809 03:32:44.095445 2670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b"
format_stamp: "Formatted at 2026-08-09 03:32:44 on dist-test-slave-gkw7"
server_key: "927ad28b381a344ee60f50140255af42"
server_key_iv: "60f0a24f9c41e635142875af3594a689"
server_key_version: "encryptionkey@0"
I20260809 03:32:44.096858 2670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:32:44.097724 2688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:44.097885 2670 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:32:44.097977 2670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b"
format_stamp: "Formatted at 2026-08-09 03:32:44 on dist-test-slave-gkw7"
server_key: "927ad28b381a344ee60f50140255af42"
server_key_iv: "60f0a24f9c41e635142875af3594a689"
server_key_version: "encryptionkey@0"
I20260809 03:32:44.098031 2670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:44.101264 2670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:44.101480 2670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:44.101646 2670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:44.101850 2670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:44.102101 2670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:44.102131 2670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:44.102150 2670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:44.102164 2670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:44.107760 2670 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:45803
I20260809 03:32:44.107843 2856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:45803 every 8 connection(s)
I20260809 03:32:44.108171 2670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:32:44.111166 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.108933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:58671 (local address 127.27.40.190:37565)
0809 03:32:44.109019 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:44.109021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:44.109234 (+ 213us) server_negotiation.cc:407] Connection header received
0809 03:32:44.109759 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:44.109760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:44.109765 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:44.109820 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:44.110930 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:44.110931 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:44.110932 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:32:44.110936 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:44.110952 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:32:44.110957 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:44.111005 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:44.111097 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 03:32:44.111111 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:32:44.111356 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.108880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37565 (local address 127.27.40.130:58671)
0809 03:32:44.109098 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:44.109147 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:44.109667 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:44.109893 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:44.109896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:44.109905 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:44.110788 (+ 883us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:32:44.110788 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:44.110816 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:32:44.110843 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:44.111017 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:44.111022 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:44.111025 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:32:44.111088 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 03:32:44.111142 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":107,"threads_started":1}
I20260809 03:32:44.111853 2860 heartbeater.cc:347] Connected to a master server at 127.27.40.190:37565
I20260809 03:32:44.111953 2860 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:44.112114 2860 heartbeater.cc:511] Master 127.27.40.190:37565 requested a full tablet report, sending...
I20260809 03:32:44.112495 2383 ts_manager.cc:194] Registered new tserver with Master: d8a4d15e81f74214aaebf61e7f2e3d1b (127.27.40.130:45803)
I20260809 03:32:44.114318 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 2670
I20260809 03:32:44.114403 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:32:44.114544 27810 external_mini_cluster.cc:1428] Setting key b850f8a112301e64cc257a3e287f8568
I20260809 03:32:44.115427 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37565
--builtin_ntp_servers=127.27.40.148:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 03:32:44.200160 2866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:44.200461 2866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:44.200502 2866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:44.201864 2866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:44.201921 2866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:44.201967 2866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:32:44.203581 2866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:44.203866 2866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:44.204335 2866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:44.209426 2874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:44.209436 2872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:44.209439 2876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:44.209699 2866 server_base.cc:1034] running on GCE node
I20260809 03:32:44.209904 2866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:44.210088 2866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:44.211266 2866 hybrid_clock.cc:648] HybridClock initialized: now 1786246364211237 us; error 40 us; skew 500 ppm
I20260809 03:32:44.212594 2866 webserver.cc:460] Webserver started at http://127.27.40.131:44089/ using document root <none> and password file <none>
I20260809 03:32:44.212867 2866 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:44.212940 2866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:44.213091 2866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:44.214215 2866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "917a994dcde04fe7b38b593dfc7c6cc7"
format_stamp: "Formatted at 2026-08-09 03:32:44 on dist-test-slave-gkw7"
server_key: "d22cb010f5c1146d78dabb89fc036ce4"
server_key_iv: "eae66ebfaadb4407301c5c8c17d6f4d3"
server_key_version: "encryptionkey@0"
I20260809 03:32:44.214581 2866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "917a994dcde04fe7b38b593dfc7c6cc7"
format_stamp: "Formatted at 2026-08-09 03:32:44 on dist-test-slave-gkw7"
server_key: "d22cb010f5c1146d78dabb89fc036ce4"
server_key_iv: "eae66ebfaadb4407301c5c8c17d6f4d3"
server_key_version: "encryptionkey@0"
I20260809 03:32:44.216023 2866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:32:44.216742 2885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:44.216913 2866 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:44.217020 2866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "917a994dcde04fe7b38b593dfc7c6cc7"
format_stamp: "Formatted at 2026-08-09 03:32:44 on dist-test-slave-gkw7"
server_key: "d22cb010f5c1146d78dabb89fc036ce4"
server_key_iv: "eae66ebfaadb4407301c5c8c17d6f4d3"
server_key_version: "encryptionkey@0"
I20260809 03:32:44.217095 2866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:44.220467 2866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:44.220731 2866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:44.220875 2866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:44.221074 2866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:44.221385 2866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:44.221442 2866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:44.221481 2866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:44.221510 2866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:44.227342 2866 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:46209
I20260809 03:32:44.227385 3051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:46209 every 8 connection(s)
I20260809 03:32:44.227713 2866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:32:44.230643 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.228458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:33657 (local address 127.27.40.190:37565)
0809 03:32:44.228535 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:44.228537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:44.228739 (+ 202us) server_negotiation.cc:407] Connection header received
0809 03:32:44.229179 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:44.229180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:44.229186 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:44.229248 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:44.230409 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:44.230410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:44.230410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:32:44.230414 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:44.230429 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:32:44.230433 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:44.230498 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:44.230547 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 03:32:44.230584 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:32:44.230808 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 2866
I20260809 03:32:44.230886 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.228379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37565 (local address 127.27.40.131:33657)
0809 03:32:44.228607 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:44.228646 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:44.229091 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:44.229300 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:44.229303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:44.229312 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:44.230267 (+ 955us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:32:44.230267 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:44.230295 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:32:44.230323 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:44.230499 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:44.230501 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:44.230503 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 03:32:44.230553 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 03:32:44.230652 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":109,"threads_started":1}
I20260809 03:32:44.230923 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:32:44.231145 27810 external_mini_cluster.cc:1428] Setting key f8069a3adfeb3e4752f091a3d62946ce
I20260809 03:32:44.231367 3052 heartbeater.cc:347] Connected to a master server at 127.27.40.190:37565
I20260809 03:32:44.231477 3052 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:44.231664 3052 heartbeater.cc:511] Master 127.27.40.190:37565 requested a full tablet report, sending...
I20260809 03:32:44.232100 2383 ts_manager.cc:194] Registered new tserver with Master: 917a994dcde04fe7b38b593dfc7c6cc7 (127.27.40.131:46209)
I20260809 03:32:44.243017 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:32:44.245620 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.244503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60958 (local address 127.27.40.190:37565)
0809 03:32:44.244561 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:44.244563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:44.244681 (+ 118us) server_negotiation.cc:407] Connection header received
0809 03:32:44.244724 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:44.244725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:44.244730 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:44.244787 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:44.245417 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:44.245417 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:44.245418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:32:44.245420 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:44.245433 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:32:44.245436 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:44.245475 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:44.245509 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 03:32:44.245521 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 03:32:44.247154 2383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:32:44.248974 2383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:44.252825 3062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.251281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:46209 (local address 127.0.0.1:51718)
0809 03:32:44.251467 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:44.251505 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:44.251570 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:44.251866 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:44.251873 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:44.251879 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:44.252390 (+ 511us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:32:44.252391 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:44.252407 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:32:44.252428 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:44.252654 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:44.252662 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:44.252665 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:32:44.252731 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 03:32:44.252755 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":90,"threads_started":1}
I20260809 03:32:44.252839 3068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.251569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51718 (local address 127.27.40.131:46209)
0809 03:32:44.251723 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:44.251726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:44.251735 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:44.251762 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:44.251768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:44.251776 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:44.251864 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:44.252514 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:44.252521 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:44.252522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:32:44.252526 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:44.252559 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:32:44.252587 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:44.252660 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:44.252736 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 03:32:44.252753 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260809 03:32:44.253419 3065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.251287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:45803 (local address 127.0.0.1:36880)
0809 03:32:44.252068 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:44.252078 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:44.252117 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:44.252584 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:44.252585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:44.252588 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:44.253020 (+ 432us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:32:44.253020 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:44.253028 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:32:44.253042 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:44.253305 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:44.253306 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:44.253306 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 03:32:44.253348 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 03:32:44.253362 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"thread_start_us":105,"threads_started":1}
I20260809 03:32:44.253641 3066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.251470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36880 (local address 127.27.40.130:45803)
0809 03:32:44.251678 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:44.251684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:44.252173 (+ 489us) server_negotiation.cc:407] Connection header received
0809 03:32:44.252203 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:44.252205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:44.252215 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:44.252288 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:44.253135 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:44.253142 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:44.253142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:32:44.253147 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:44.253186 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:32:44.253208 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:44.253277 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:44.253515 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 03:32:44.253559 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":106,"threads_started":1}
I20260809 03:32:44.253706 2957 tablet_service.cc:1467] Processing CreateTablet for tablet 15923aade62449249675c42f37d0631d (DEFAULT_TABLE table=test-table [id=fe31cbc0deca4d478cac148f954d496b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:44.254069 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.251281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:44663 (local address 127.0.0.1:50264)
0809 03:32:44.251907 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:44.251918 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:44.251976 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:44.252720 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:44.252721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:44.252723 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:44.253378 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:32:44.253379 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:44.253386 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:32:44.253399 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:44.253938 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:44.253939 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:44.253939 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 03:32:44.253993 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 03:32:44.254007 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260809 03:32:44.254082 2957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15923aade62449249675c42f37d0631d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:44.254356 3069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.251699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50264 (local address 127.27.40.129:44663)
0809 03:32:44.251893 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:44.251905 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:44.251970 (+ 65us) server_negotiation.cc:407] Connection header received
0809 03:32:44.252031 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:44.252034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:44.252045 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:44.252446 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:44.253588 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:44.253596 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:44.253596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:32:44.253601 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:44.253634 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:32:44.253669 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:44.253749 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:44.254245 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 03:32:44.254264 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
I20260809 03:32:44.254309 2755 tablet_service.cc:1467] Processing CreateTablet for tablet 15923aade62449249675c42f37d0631d (DEFAULT_TABLE table=test-table [id=fe31cbc0deca4d478cac148f954d496b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:44.254665 2755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15923aade62449249675c42f37d0631d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:44.255008 2568 tablet_service.cc:1467] Processing CreateTablet for tablet 15923aade62449249675c42f37d0631d (DEFAULT_TABLE table=test-table [id=fe31cbc0deca4d478cac148f954d496b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:44.255348 2568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15923aade62449249675c42f37d0631d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:44.257143 3071 tablet_bootstrap.cc:492] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7: Bootstrap starting.
I20260809 03:32:44.257922 3071 tablet_bootstrap.cc:654] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:44.258405 3071 log.cc:824] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:44.258462 3072 tablet_bootstrap.cc:492] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b: Bootstrap starting.
I20260809 03:32:44.258764 3074 tablet_bootstrap.cc:492] T 15923aade62449249675c42f37d0631d P 35b34fbea0e64ffd8ec3f6fd6100fd67: Bootstrap starting.
I20260809 03:32:44.259871 3074 tablet_bootstrap.cc:654] T 15923aade62449249675c42f37d0631d P 35b34fbea0e64ffd8ec3f6fd6100fd67: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:44.259956 3072 tablet_bootstrap.cc:654] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:44.260515 3074 log.cc:824] T 15923aade62449249675c42f37d0631d P 35b34fbea0e64ffd8ec3f6fd6100fd67: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:44.260584 3072 log.cc:824] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:44.261404 3071 tablet_bootstrap.cc:492] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7: No bootstrap required, opened a new log
I20260809 03:32:44.261493 3072 tablet_bootstrap.cc:492] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b: No bootstrap required, opened a new log
I20260809 03:32:44.261492 3074 tablet_bootstrap.cc:492] T 15923aade62449249675c42f37d0631d P 35b34fbea0e64ffd8ec3f6fd6100fd67: No bootstrap required, opened a new log
I20260809 03:32:44.261492 3071 ts_tablet_manager.cc:1397] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260809 03:32:44.261590 3072 ts_tablet_manager.cc:1397] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:32:44.261587 3074 ts_tablet_manager.cc:1397] T 15923aade62449249675c42f37d0631d P 35b34fbea0e64ffd8ec3f6fd6100fd67: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:32:44.264349 3071 raft_consensus.cc:348] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45803 } } peers { permanent_uuid: "35b34fbea0e64ffd8ec3f6fd6100fd67" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44663 } } peers { permanent_uuid: "917a994dcde04fe7b38b593dfc7c6cc7" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46209 } }
I20260809 03:32:44.264510 3071 raft_consensus.cc:374] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:44.264555 3071 raft_consensus.cc:729] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 917a994dcde04fe7b38b593dfc7c6cc7, State: Initialized, Role: FOLLOWER
I20260809 03:32:44.264698 3071 consensus_queue.cc:260] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45803 } } peers { permanent_uuid: "35b34fbea0e64ffd8ec3f6fd6100fd67" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44663 } } peers { permanent_uuid: "917a994dcde04fe7b38b593dfc7c6cc7" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46209 } }
I20260809 03:32:44.264962 3071 ts_tablet_manager.cc:1428] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260809 03:32:44.265111 3052 heartbeater.cc:503] Master 127.27.40.190:37565 was elected leader, sending a full tablet report...
I20260809 03:32:44.265477 3074 raft_consensus.cc:348] T 15923aade62449249675c42f37d0631d P 35b34fbea0e64ffd8ec3f6fd6100fd67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45803 } } peers { permanent_uuid: "35b34fbea0e64ffd8ec3f6fd6100fd67" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44663 } } peers { permanent_uuid: "917a994dcde04fe7b38b593dfc7c6cc7" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46209 } }
I20260809 03:32:44.265679 3074 raft_consensus.cc:374] T 15923aade62449249675c42f37d0631d P 35b34fbea0e64ffd8ec3f6fd6100fd67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:44.265715 3074 raft_consensus.cc:729] T 15923aade62449249675c42f37d0631d P 35b34fbea0e64ffd8ec3f6fd6100fd67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b34fbea0e64ffd8ec3f6fd6100fd67, State: Initialized, Role: FOLLOWER
I20260809 03:32:44.265652 3072 raft_consensus.cc:348] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45803 } } peers { permanent_uuid: "35b34fbea0e64ffd8ec3f6fd6100fd67" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44663 } } peers { permanent_uuid: "917a994dcde04fe7b38b593dfc7c6cc7" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46209 } }
I20260809 03:32:44.265787 3072 raft_consensus.cc:374] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:44.265828 3072 raft_consensus.cc:729] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8a4d15e81f74214aaebf61e7f2e3d1b, State: Initialized, Role: FOLLOWER
I20260809 03:32:44.265884 3074 consensus_queue.cc:260] T 15923aade62449249675c42f37d0631d P 35b34fbea0e64ffd8ec3f6fd6100fd67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45803 } } peers { permanent_uuid: "35b34fbea0e64ffd8ec3f6fd6100fd67" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44663 } } peers { permanent_uuid: "917a994dcde04fe7b38b593dfc7c6cc7" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46209 } }
I20260809 03:32:44.265933 3072 consensus_queue.cc:260] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45803 } } peers { permanent_uuid: "35b34fbea0e64ffd8ec3f6fd6100fd67" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44663 } } peers { permanent_uuid: "917a994dcde04fe7b38b593dfc7c6cc7" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46209 } }
I20260809 03:32:44.266177 3074 ts_tablet_manager.cc:1428] T 15923aade62449249675c42f37d0631d P 35b34fbea0e64ffd8ec3f6fd6100fd67: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:32:44.266196 3072 ts_tablet_manager.cc:1428] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260809 03:32:44.266263 2860 heartbeater.cc:503] Master 127.27.40.190:37565 was elected leader, sending a full tablet report...
I20260809 03:32:44.266299 2665 heartbeater.cc:503] Master 127.27.40.190:37565 was elected leader, sending a full tablet report...
W20260809 03:32:44.358811 2861 tablet.cc:2369] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:32:44.478242 3053 tablet.cc:2369] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:32:44.481809 3083 raft_consensus.cc:482] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:44.481927 3083 raft_consensus.cc:504] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45803 } } peers { permanent_uuid: "35b34fbea0e64ffd8ec3f6fd6100fd67" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44663 } } peers { permanent_uuid: "917a994dcde04fe7b38b593dfc7c6cc7" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46209 } }
I20260809 03:32:44.482316 3083 leader_election.cc:288] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35b34fbea0e64ffd8ec3f6fd6100fd67 (127.27.40.129:44663), 917a994dcde04fe7b38b593dfc7c6cc7 (127.27.40.131:46209)
I20260809 03:32:44.483860 3068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.482640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:35507 (local address 127.27.40.131:46209)
0809 03:32:44.482692 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:44.482695 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:44.482705 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:32:44.482734 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:44.482735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:44.482740 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:44.482800 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:44.483540 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:44.483543 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:44.483543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:32:44.483549 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:44.483568 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:32:44.483573 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:44.483671 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:44.483771 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 03:32:44.483788 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 03:32:44.483860 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.482515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:46209 (local address 127.27.40.130:35507)
0809 03:32:44.482589 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:44.482602 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:44.482647 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:44.482841 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:44.482843 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:44.482846 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:44.483403 (+ 557us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:32:44.483404 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:44.483415 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:32:44.483430 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:44.483697 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:44.483698 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:44.483699 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 03:32:44.483760 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 03:32:44.483777 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 03:32:44.484063 3069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.482603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:45741 (local address 127.27.40.129:44663)
0809 03:32:44.482666 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:44.482668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:44.482915 (+ 247us) server_negotiation.cc:407] Connection header received
0809 03:32:44.482966 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:44.482967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:44.482972 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:44.483041 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:44.483742 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:44.483743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:44.483744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:32:44.483749 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:44.483765 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:32:44.483771 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:44.483864 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:44.483938 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 03:32:44.483968 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 03:32:44.484253 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.482570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:44663 (local address 127.27.40.130:45741)
0809 03:32:44.482830 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:44.482841 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:44.482910 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:44.483050 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:44.483051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:44.483055 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:44.483653 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:32:44.483654 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:44.483661 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:32:44.483674 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:44.483861 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:44.483862 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:44.483863 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 03:32:44.484170 (+ 307us) client_negotiation.cc:240] Negotiation successful
0809 03:32:44.484184 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":121,"threads_started":1}
I20260809 03:32:44.484319 2986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15923aade62449249675c42f37d0631d" candidate_uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917a994dcde04fe7b38b593dfc7c6cc7" is_pre_election: true
I20260809 03:32:44.484488 2986 raft_consensus.cc:2454] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8a4d15e81f74214aaebf61e7f2e3d1b in term 0.
I20260809 03:32:44.484539 2599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15923aade62449249675c42f37d0631d" candidate_uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b34fbea0e64ffd8ec3f6fd6100fd67" is_pre_election: true
I20260809 03:32:44.484705 2599 raft_consensus.cc:2454] T 15923aade62449249675c42f37d0631d P 35b34fbea0e64ffd8ec3f6fd6100fd67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8a4d15e81f74214aaebf61e7f2e3d1b in term 0.
I20260809 03:32:44.484802 2692 leader_election.cc:302] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 917a994dcde04fe7b38b593dfc7c6cc7, d8a4d15e81f74214aaebf61e7f2e3d1b; no voters:
I20260809 03:32:44.484937 3083 raft_consensus.cc:2781] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:44.484994 3083 raft_consensus.cc:482] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:44.485049 3083 raft_consensus.cc:3037] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:44.486244 3083 raft_consensus.cc:504] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45803 } } peers { permanent_uuid: "35b34fbea0e64ffd8ec3f6fd6100fd67" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44663 } } peers { permanent_uuid: "917a994dcde04fe7b38b593dfc7c6cc7" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46209 } }
I20260809 03:32:44.486387 3083 leader_election.cc:288] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [CANDIDATE]: Term 1 election: Requested vote from peers 35b34fbea0e64ffd8ec3f6fd6100fd67 (127.27.40.129:44663), 917a994dcde04fe7b38b593dfc7c6cc7 (127.27.40.131:46209)
I20260809 03:32:44.486568 2599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15923aade62449249675c42f37d0631d" candidate_uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b34fbea0e64ffd8ec3f6fd6100fd67"
I20260809 03:32:44.486667 2599 raft_consensus.cc:3037] T 15923aade62449249675c42f37d0631d P 35b34fbea0e64ffd8ec3f6fd6100fd67 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:44.487747 2599 raft_consensus.cc:2454] T 15923aade62449249675c42f37d0631d P 35b34fbea0e64ffd8ec3f6fd6100fd67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8a4d15e81f74214aaebf61e7f2e3d1b in term 1.
I20260809 03:32:44.487884 2986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15923aade62449249675c42f37d0631d" candidate_uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917a994dcde04fe7b38b593dfc7c6cc7"
I20260809 03:32:44.487974 2986 raft_consensus.cc:3037] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:44.488047 2694 leader_election.cc:302] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b34fbea0e64ffd8ec3f6fd6100fd67, d8a4d15e81f74214aaebf61e7f2e3d1b; no voters:
I20260809 03:32:44.488173 3083 raft_consensus.cc:2781] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:44.488351 3083 raft_consensus.cc:686] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [term 1 LEADER]: Becoming Leader. State: Replica: d8a4d15e81f74214aaebf61e7f2e3d1b, State: Running, Role: LEADER
I20260809 03:32:44.488474 3083 consensus_queue.cc:237] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45803 } } peers { permanent_uuid: "35b34fbea0e64ffd8ec3f6fd6100fd67" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44663 } } peers { permanent_uuid: "917a994dcde04fe7b38b593dfc7c6cc7" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46209 } }
I20260809 03:32:44.489125 2986 raft_consensus.cc:2454] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8a4d15e81f74214aaebf61e7f2e3d1b in term 1.
I20260809 03:32:44.489758 2380 catalog_manager.cc:5495] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b reported cstate change: term changed from 0 to 1, leader changed from <none> to d8a4d15e81f74214aaebf61e7f2e3d1b (127.27.40.130). New cstate: current_term: 1 leader_uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a4d15e81f74214aaebf61e7f2e3d1b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35b34fbea0e64ffd8ec3f6fd6100fd67" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "917a994dcde04fe7b38b593dfc7c6cc7" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46209 } health_report { overall_health: UNKNOWN } } }
W20260809 03:32:44.493009 2666 tablet.cc:2369] T 15923aade62449249675c42f37d0631d P 35b34fbea0e64ffd8ec3f6fd6100fd67: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:32:44.537546 3066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.536184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36888 (local address 127.27.40.130:45803)
0809 03:32:44.536270 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:44.536272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:44.536284 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:32:44.536338 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:44.536339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:44.536355 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:44.536416 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:44.537229 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:44.537230 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:44.537230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:32:44.537235 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:44.537252 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:32:44.537256 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:44.537305 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:44.537425 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 03:32:44.537442 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 03:32:44.539801 2599 raft_consensus.cc:1261] T 15923aade62449249675c42f37d0631d P 35b34fbea0e64ffd8ec3f6fd6100fd67 [term 1 FOLLOWER]: Refusing update from remote peer d8a4d15e81f74214aaebf61e7f2e3d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:44.539801 2986 raft_consensus.cc:1261] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7 [term 1 FOLLOWER]: Refusing update from remote peer d8a4d15e81f74214aaebf61e7f2e3d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:44.540236 3085 consensus_queue.cc:1035] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b34fbea0e64ffd8ec3f6fd6100fd67" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:44.540427 3083 consensus_queue.cc:1035] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "917a994dcde04fe7b38b593dfc7c6cc7" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:44.543982 3091 mvcc.cc:205] Tried to move back new op lower bound from 7316465109151383552 to 7316465108945383424. Current Snapshot: MvccSnapshot[applied={T|T < 7316465109151383552 or (T in {7316465109151383552})}]
I20260809 03:32:44.546440 2383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:44.546576 2383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:44.547427 2383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 470eb8ef1094461ebb42fc79b9a0a5c3: Sending DeleteTablet for 3 replicas of tablet 15923aade62449249675c42f37d0631d
I20260809 03:32:44.547775 2755 tablet_service.cc:1514] Processing DeleteTablet for tablet 15923aade62449249675c42f37d0631d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:44 UTC) from {username='slave'} at 127.0.0.1:36880
I20260809 03:32:44.547806 2957 tablet_service.cc:1514] Processing DeleteTablet for tablet 15923aade62449249675c42f37d0631d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:44 UTC) from {username='slave'} at 127.0.0.1:51718
I20260809 03:32:44.548041 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 2474
I20260809 03:32:44.552261 3109 tablet_replica.cc:330] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b: stopping tablet replica
I20260809 03:32:44.552409 3109 raft_consensus.cc:2229] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:32:44.552580 3109 raft_consensus.cc:2258] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:44.553704 3108 tablet_replica.cc:330] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7: stopping tablet replica
I20260809 03:32:44.553821 3108 raft_consensus.cc:2229] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:44.553920 3108 raft_consensus.cc:2258] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:44.554828 3108 ts_tablet_manager.cc:1905] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:44.555785 3109 ts_tablet_manager.cc:1905] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:44.556808 3108 ts_tablet_manager.cc:1918] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:44.556900 3108 log.cc:1196] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/15923aade62449249675c42f37d0631d
I20260809 03:32:44.557193 3108 ts_tablet_manager.cc:1939] T 15923aade62449249675c42f37d0631d P 917a994dcde04fe7b38b593dfc7c6cc7: Deleting consensus metadata
I20260809 03:32:44.557792 2364 catalog_manager.cc:4841] TS 917a994dcde04fe7b38b593dfc7c6cc7 (127.27.40.131:46209): tablet 15923aade62449249675c42f37d0631d (table test-table [id=fe31cbc0deca4d478cac148f954d496b]) successfully deleted
I20260809 03:32:44.558173 3109 ts_tablet_manager.cc:1918] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:44.558276 3109 log.cc:1196] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/15923aade62449249675c42f37d0631d
I20260809 03:32:44.558561 3109 ts_tablet_manager.cc:1939] T 15923aade62449249675c42f37d0631d P d8a4d15e81f74214aaebf61e7f2e3d1b: Deleting consensus metadata
I20260809 03:32:44.559011 2362 catalog_manager.cc:4841] TS d8a4d15e81f74214aaebf61e7f2e3d1b (127.27.40.130:45803): tablet 15923aade62449249675c42f37d0631d (table test-table [id=fe31cbc0deca4d478cac148f954d496b]) successfully deleted
W20260809 03:32:44.574024 2365 connection.cc:535] client connection to 127.27.40.129:44663 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 03:32:44.574290 2365 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 03:32:44.574426 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 2670
I20260809 03:32:44.578001 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.577800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:44663 (local address 127.0.0.1:50266)
0809 03:32:44.577881 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:44.577949 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:44663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 03:32:44.578178 2365 catalog_manager.cc:4584] TS 35b34fbea0e64ffd8ec3f6fd6100fd67 (127.27.40.129:44663): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 15923aade62449249675c42f37d0631d: Network error: Client connection negotiation failed: client connection to 127.27.40.129:44663: connect: Connection refused (error 111)
I20260809 03:32:44.583700 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 2866
I20260809 03:32:44.597470 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 2334
2026-08-09T03:32:44Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (890 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 03:32:44.623553 27810 mini_oidc.cc:152] Starting JWKS server
I20260809 03:32:44.624688 27810 webserver.cc:460] Webserver started at https://127.0.0.1:44441/ using document root <none> and password file <none>
I20260809 03:32:44.624814 27810 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 03:32:44.625000 27810 webserver.cc:460] Webserver started at http://127.0.0.1:35487/ using document root <none> and password file <none>
2026-08-09T03:32:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:44Z Disabled control of system clock
I20260809 03:32:44.633912 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:36217
--webserver_interface=127.27.40.190
--webserver_port=0
--builtin_ntp_servers=127.27.40.148:40489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:36217
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44441/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/testchainca.cert with env {}
W20260809 03:32:44.717414 3116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:44.717700 3116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:44.717727 3116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:44.718909 3116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:44.718954 3116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:44.718967 3116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:44.718981 3116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:44.718992 3116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 03:32:44.719004 3116 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 03:32:44.719017 3116 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44441/jwks.json
I20260809 03:32:44.720623 3116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:40489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:36217
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:36217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44441/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:44.720892 3116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:44.721120 3116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:44.726222 3128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:44.726204 3127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:44.726204 3131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:44.726547 3116 server_base.cc:1034] running on GCE node
I20260809 03:32:44.726804 3116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:44.727061 3116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:44.728223 3116 hybrid_clock.cc:648] HybridClock initialized: now 1786246364728209 us; error 48 us; skew 500 ppm
I20260809 03:32:44.729574 3116 webserver.cc:460] Webserver started at http://127.27.40.190:34621/ using document root <none> and password file <none>
I20260809 03:32:44.729861 3116 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:44.729933 3116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:44.730083 3116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:44.731169 3116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "010cd6e776984396a4b40a466b04029b"
format_stamp: "Formatted at 2026-08-09 03:32:44 on dist-test-slave-gkw7"
server_key: "8e6d3e48495ed52c6f7816936ccf6166"
server_key_iv: "b0b54f7a7d102137f0ad8c4e53091011"
server_key_version: "encryptionkey@0"
I20260809 03:32:44.731529 3116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "010cd6e776984396a4b40a466b04029b"
format_stamp: "Formatted at 2026-08-09 03:32:44 on dist-test-slave-gkw7"
server_key: "8e6d3e48495ed52c6f7816936ccf6166"
server_key_iv: "b0b54f7a7d102137f0ad8c4e53091011"
server_key_version: "encryptionkey@0"
I20260809 03:32:44.732810 3116 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:32:44.733496 3141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:44.733677 3116 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:44.733788 3116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "010cd6e776984396a4b40a466b04029b"
format_stamp: "Formatted at 2026-08-09 03:32:44 on dist-test-slave-gkw7"
server_key: "8e6d3e48495ed52c6f7816936ccf6166"
server_key_iv: "b0b54f7a7d102137f0ad8c4e53091011"
server_key_version: "encryptionkey@0"
I20260809 03:32:44.733862 3116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:44.755396 3116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:44.755782 3116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:44.755961 3116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:44.760306 3116 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:36217
I20260809 03:32:44.760355 3218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:36217 every 8 connection(s)
I20260809 03:32:44.760733 3116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:44.761875 3220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:44.764309 3220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b: Bootstrap starting.
I20260809 03:32:44.764945 3220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:44.765267 3220 log.cc:824] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:44.766021 3220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b: No bootstrap required, opened a new log
I20260809 03:32:44.767616 3220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010cd6e776984396a4b40a466b04029b" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36217 } }
I20260809 03:32:44.767763 3220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:44.767787 3220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010cd6e776984396a4b40a466b04029b, State: Initialized, Role: FOLLOWER
I20260809 03:32:44.767963 3220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010cd6e776984396a4b40a466b04029b" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36217 } }
I20260809 03:32:44.768051 3220 raft_consensus.cc:388] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:44.768106 3220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:44.768159 3220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:44.768656 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 3116
I20260809 03:32:44.768702 3220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010cd6e776984396a4b40a466b04029b" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36217 } }
I20260809 03:32:44.768775 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:44.768810 3220 leader_election.cc:302] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 010cd6e776984396a4b40a466b04029b; no voters:
I20260809 03:32:44.768985 27810 external_mini_cluster.cc:1428] Setting key a44714626374ff0645523cb946e54b4c
I20260809 03:32:44.768983 3220 leader_election.cc:288] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:44.769017 3225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:44.769160 3225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [term 1 LEADER]: Becoming Leader. State: Replica: 010cd6e776984396a4b40a466b04029b, State: Running, Role: LEADER
I20260809 03:32:44.769256 3225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010cd6e776984396a4b40a466b04029b" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36217 } }
I20260809 03:32:44.769287 3220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:44.770256 3229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 010cd6e776984396a4b40a466b04029b. Latest consensus state: current_term: 1 leader_uuid: "010cd6e776984396a4b40a466b04029b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010cd6e776984396a4b40a466b04029b" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36217 } } }
I20260809 03:32:44.770344 3227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "010cd6e776984396a4b40a466b04029b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010cd6e776984396a4b40a466b04029b" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36217 } } }
I20260809 03:32:44.770444 3227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:44.770359 3229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:44.770764 3242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:44.771405 3242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:44.773315 3242 catalog_manager.cc:1347] Generated new cluster ID: 6a80c9ad54bb4535b7cbb9964494046d
I20260809 03:32:44.773373 3242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:32:44.775411 3232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.769803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34242 (local address 127.27.40.190:36217)
0809 03:32:44.770058 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:44.770067 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:44.770107 (+ 40us) server_negotiation.cc:407] Connection header received
0809 03:32:44.770231 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:44.770239 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:44.770332 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:44.770483 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:44.770867 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:44.772171 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:44.773257 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:44.773806 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:44.773943 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:44.774726 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:44.774730 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:44.774731 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:32:44.774913 (+ 182us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:44.774945 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:32:44.774955 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:44.775032 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:44.775106 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 03:32:44.775191 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":150,"threads_started":1}
I20260809 03:32:44.776264 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:32:44.796149 3242 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:44.796907 3242 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:44.803053 3242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 010cd6e776984396a4b40a466b04029b: Generated new TSK 0
I20260809 03:32:44.803246 3242 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 03:32:44.830767 3268 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 03:32:44.830933 3268 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 03:32:44.831166 3232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:44.814025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34258 (local address 127.27.40.190:36217)
0809 03:32:44.814087 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:44.814089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:44.814245 (+ 156us) server_negotiation.cc:407] Connection header received
0809 03:32:44.814315 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:44.814316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:44.814329 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:44.814414 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0809 03:32:44.814654 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:44.815445 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:44.816300 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:44.816628 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:44.816698 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:44.816953 (+ 255us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 03:32:44.830931 (+ 13978us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 03:32:44.831093 (+ 162us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34258: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 03:32:44.831230 3232 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34258: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 03:32:44.831588 3267 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:32:44.813942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:36217 (local address 127.0.0.1:34258)
0809 03:32:44.814154 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:44.814164 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:44.814239 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:44.814462 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:44.814464 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:44.814466 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 03:32:44.814556 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:44.814559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:44.815553 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:44.815555 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:44.816211 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:44.816214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:44.816740 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:44.816742 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:44.816815 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:44.816836 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 03:32:44.831108 (+ 14272us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 03:32:44.831256 (+ 148us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.40.190:36217: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":102,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.27.40.190:36217: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 03:32:44.832190 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 3116
2026-08-09T03:32:44Z chronyd exiting
I20260809 03:32:44.872352 27810 test_util.cc:175] -----------------------------------------------
I20260809 03:32:44.872448 27810 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246339702443-27810-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (250 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 03:32:44.873337 27810 mini_oidc.cc:152] Starting JWKS server
I20260809 03:32:44.874153 27810 webserver.cc:460] Webserver started at https://127.0.0.1:34503/ using document root <none> and password file <none>
I20260809 03:32:44.874202 27810 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 03:32:44.874389 27810 webserver.cc:460] Webserver started at http://127.0.0.1:44219/ using document root <none> and password file <none>
2026-08-09T03:32:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:44Z Disabled control of system clock
I20260809 03:32:44.883579 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:40841
--webserver_interface=127.27.40.190
--webserver_port=0
--builtin_ntp_servers=127.27.40.148:43657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:40841
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34503/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/testchainca.cert with env {}
W20260809 03:32:44.965116 3276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:44.965315 3276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:44.965337 3276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:44.966562 3276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:44.966609 3276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:44.966621 3276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:44.966634 3276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:44.966647 3276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 03:32:44.966660 3276 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 03:32:44.966713 3276 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34503/jwks.json
I20260809 03:32:44.968207 3276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:43657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:40841
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:40841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34503/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:44.968461 3276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:44.968672 3276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:44.973642 3284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:44.973727 3295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:44.973663 3286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:44.973812 3276 server_base.cc:1034] running on GCE node
I20260809 03:32:44.974123 3276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:44.974427 3276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:44.975617 3276 hybrid_clock.cc:648] HybridClock initialized: now 1786246364975582 us; error 51 us; skew 500 ppm
I20260809 03:32:44.977032 3276 webserver.cc:460] Webserver started at http://127.27.40.190:43853/ using document root <none> and password file <none>
I20260809 03:32:44.977283 3276 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:44.977329 3276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:44.977466 3276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:44.978650 3276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "9b38ab3646b5401e91a0c0d5de123880"
format_stamp: "Formatted at 2026-08-09 03:32:44 on dist-test-slave-gkw7"
server_key: "0a47d873e85f6debfa13142052c9361b"
server_key_iv: "d1de76574436656d5ac0fcfc004de75c"
server_key_version: "encryptionkey@0"
I20260809 03:32:44.979022 3276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "9b38ab3646b5401e91a0c0d5de123880"
format_stamp: "Formatted at 2026-08-09 03:32:44 on dist-test-slave-gkw7"
server_key: "0a47d873e85f6debfa13142052c9361b"
server_key_iv: "d1de76574436656d5ac0fcfc004de75c"
server_key_version: "encryptionkey@0"
I20260809 03:32:44.980420 3276 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:32:44.981261 3301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:44.981496 3276 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:44.981633 3276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "9b38ab3646b5401e91a0c0d5de123880"
format_stamp: "Formatted at 2026-08-09 03:32:44 on dist-test-slave-gkw7"
server_key: "0a47d873e85f6debfa13142052c9361b"
server_key_iv: "d1de76574436656d5ac0fcfc004de75c"
server_key_version: "encryptionkey@0"
I20260809 03:32:44.981732 3276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:45.001873 3276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:45.002219 3276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:45.002382 3276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:45.006794 3276 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:40841
I20260809 03:32:45.006865 3384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:40841 every 8 connection(s)
I20260809 03:32:45.007242 3276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:45.008004 3388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:45.008214 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 3276
I20260809 03:32:45.008353 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:45.008641 27810 external_mini_cluster.cc:1428] Setting key 206df259c27547c1d0393e0a78e31c31
I20260809 03:32:45.011693 3388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880: Bootstrap starting.
I20260809 03:32:45.012336 3388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:45.012673 3388 log.cc:824] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:45.013645 3388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880: No bootstrap required, opened a new log
I20260809 03:32:45.014370 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:45.009200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60052 (local address 127.27.40.190:40841)
0809 03:32:45.009396 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:45.009403 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:45.009427 (+ 24us) server_negotiation.cc:407] Connection header received
0809 03:32:45.009870 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:45.009877 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:45.009938 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:45.010091 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:45.010335 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.011210 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.012078 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.012545 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.012685 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:45.013568 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:45.013573 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:45.013574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:32:45.013801 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:45.013858 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:32:45.013870 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:45.013931 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:45.014065 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 03:32:45.014174 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260809 03:32:45.015486 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:32:45.015970 3388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b38ab3646b5401e91a0c0d5de123880" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40841 } }
I20260809 03:32:45.016155 3388 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:45.016196 3388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b38ab3646b5401e91a0c0d5de123880, State: Initialized, Role: FOLLOWER
I20260809 03:32:45.016324 3388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b38ab3646b5401e91a0c0d5de123880" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40841 } }
I20260809 03:32:45.016431 3388 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:45.016476 3388 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:45.016531 3388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:45.017132 3388 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b38ab3646b5401e91a0c0d5de123880" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40841 } }
I20260809 03:32:45.017249 3388 leader_election.cc:302] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b38ab3646b5401e91a0c0d5de123880; no voters:
I20260809 03:32:45.017431 3388 leader_election.cc:288] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:45.017488 3394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:45.017709 3394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [term 1 LEADER]: Becoming Leader. State: Replica: 9b38ab3646b5401e91a0c0d5de123880, State: Running, Role: LEADER
I20260809 03:32:45.017719 3388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:45.017841 3394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b38ab3646b5401e91a0c0d5de123880" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40841 } }
I20260809 03:32:45.018249 3395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b38ab3646b5401e91a0c0d5de123880" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b38ab3646b5401e91a0c0d5de123880" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40841 } } }
I20260809 03:32:45.018368 3395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:45.018249 3396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b38ab3646b5401e91a0c0d5de123880. Latest consensus state: current_term: 1 leader_uuid: "9b38ab3646b5401e91a0c0d5de123880" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b38ab3646b5401e91a0c0d5de123880" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40841 } } }
I20260809 03:32:45.018520 3396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880 [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:45.018639 3410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:45.019564 3410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:45.021288 3410 catalog_manager.cc:1347] Generated new cluster ID: d0f60c5e5b1f40c087d0fcbe582f34ee
I20260809 03:32:45.021358 3410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:32:45.034749 3410 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:45.035374 3410 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:45.040632 3410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b38ab3646b5401e91a0c0d5de123880: Generated new TSK 0
I20260809 03:32:45.040829 3410 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 03:32:45.060322 3438 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 03:32:45.060413 3438 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 03:32:45.060743 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:45.045838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60054 (local address 127.27.40.190:40841)
0809 03:32:45.045903 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:45.045906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:45.046042 (+ 136us) server_negotiation.cc:407] Connection header received
0809 03:32:45.046138 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:45.046138 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:45.046150 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:45.046211 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0809 03:32:45.046398 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.046886 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.048009 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.048205 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.048257 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:45.048463 (+ 206us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 03:32:45.060558 (+ 12095us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34503/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 03:32:45.060692 (+ 134us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60054: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34503/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 03:32:45.060817 3392 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60054: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34503/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 03:32:45.061079 3437 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:32:45.045771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40841 (local address 127.0.0.1:60054)
0809 03:32:45.045961 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:45.045975 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:45.046054 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:45.046242 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:45.046243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:45.046245 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 03:32:45.046315 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:45.046318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.046976 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.046977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:45.047917 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:45.047921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.048292 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.048293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:45.048357 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:45.048364 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 03:32:45.060692 (+ 12328us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34503/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 03:32:45.060858 (+ 166us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.40.190:40841: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34503/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":96,"threads_started":1}
I20260809 03:32:45.061504 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 3276
2026-08-09T03:32:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (234 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 03:32:45.108260 27810 mini_oidc.cc:152] Starting JWKS server
I20260809 03:32:45.108989 27810 webserver.cc:460] Webserver started at https://127.0.0.1:34561/ using document root <none> and password file <none>
I20260809 03:32:45.109061 27810 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 03:32:45.109247 27810 webserver.cc:460] Webserver started at http://127.0.0.1:35543/ using document root <none> and password file <none>
2026-08-09T03:32:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:45Z Disabled control of system clock
I20260809 03:32:45.118729 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:43979
--webserver_interface=127.27.40.190
--webserver_port=0
--builtin_ntp_servers=127.27.40.148:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:43979
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34561/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 03:32:45.202040 3451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:45.202255 3451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:45.202276 3451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:45.203372 3451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:45.203404 3451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:45.203418 3451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:45.203429 3451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:45.203441 3451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 03:32:45.203454 3451 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 03:32:45.203465 3451 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34561/jwks.json
I20260809 03:32:45.204746 3451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:43979
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:43979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34561/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:45.204934 3451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:45.205166 3451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:45.210142 3459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:45.210162 3463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:45.210184 3458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:45.210539 3451 server_base.cc:1034] running on GCE node
I20260809 03:32:45.210762 3451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:45.211015 3451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:45.212168 3451 hybrid_clock.cc:648] HybridClock initialized: now 1786246365212152 us; error 31 us; skew 500 ppm
I20260809 03:32:45.213420 3451 webserver.cc:460] Webserver started at http://127.27.40.190:44813/ using document root <none> and password file <none>
I20260809 03:32:45.213687 3451 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:45.213747 3451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:45.213888 3451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:45.214965 3451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "60d4a8e90e31489d88d8cb30df329feb"
format_stamp: "Formatted at 2026-08-09 03:32:45 on dist-test-slave-gkw7"
server_key: "ef881f5182efc58e61a36c1abacb09a6"
server_key_iv: "9670b7beea6e4e3e769ac3f5e4aaa39f"
server_key_version: "encryptionkey@0"
I20260809 03:32:45.215333 3451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "60d4a8e90e31489d88d8cb30df329feb"
format_stamp: "Formatted at 2026-08-09 03:32:45 on dist-test-slave-gkw7"
server_key: "ef881f5182efc58e61a36c1abacb09a6"
server_key_iv: "9670b7beea6e4e3e769ac3f5e4aaa39f"
server_key_version: "encryptionkey@0"
I20260809 03:32:45.216648 3451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:32:45.217339 3470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:45.217514 3451 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:45.217660 3451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "60d4a8e90e31489d88d8cb30df329feb"
format_stamp: "Formatted at 2026-08-09 03:32:45 on dist-test-slave-gkw7"
server_key: "ef881f5182efc58e61a36c1abacb09a6"
server_key_iv: "9670b7beea6e4e3e769ac3f5e4aaa39f"
server_key_version: "encryptionkey@0"
I20260809 03:32:45.217744 3451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:45.233520 3451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:45.233912 3451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:45.234076 3451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:45.238449 3451 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:43979
I20260809 03:32:45.238494 3552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:43979 every 8 connection(s)
I20260809 03:32:45.238862 3451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:45.239632 3554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:45.242337 3554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb: Bootstrap starting.
I20260809 03:32:45.242892 3554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:45.243134 3554 log.cc:824] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:45.243517 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 3451
I20260809 03:32:45.243604 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:45.243839 27810 external_mini_cluster.cc:1428] Setting key c5a2357ba8c5efa44b89463090e1238c
I20260809 03:32:45.244117 3554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb: No bootstrap required, opened a new log
I20260809 03:32:45.246016 3554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d4a8e90e31489d88d8cb30df329feb" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 43979 } }
I20260809 03:32:45.246153 3554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:45.246176 3554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60d4a8e90e31489d88d8cb30df329feb, State: Initialized, Role: FOLLOWER
I20260809 03:32:45.246330 3554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d4a8e90e31489d88d8cb30df329feb" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 43979 } }
I20260809 03:32:45.246426 3554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:45.246470 3554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:45.246531 3554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:45.247280 3554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d4a8e90e31489d88d8cb30df329feb" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 43979 } }
I20260809 03:32:45.247387 3554 leader_election.cc:302] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60d4a8e90e31489d88d8cb30df329feb; no voters:
I20260809 03:32:45.247534 3554 leader_election.cc:288] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:45.247634 3562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:45.247782 3554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:45.247816 3562 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [term 1 LEADER]: Becoming Leader. State: Replica: 60d4a8e90e31489d88d8cb30df329feb, State: Running, Role: LEADER
I20260809 03:32:45.247972 3562 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d4a8e90e31489d88d8cb30df329feb" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 43979 } }
I20260809 03:32:45.248404 3563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60d4a8e90e31489d88d8cb30df329feb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d4a8e90e31489d88d8cb30df329feb" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 43979 } } }
I20260809 03:32:45.248394 3564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60d4a8e90e31489d88d8cb30df329feb. Latest consensus state: current_term: 1 leader_uuid: "60d4a8e90e31489d88d8cb30df329feb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d4a8e90e31489d88d8cb30df329feb" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 43979 } } }
I20260809 03:32:45.248497 3563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:45.248502 3564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:45.248838 3574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:45.249380 3574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:45.251044 3574 catalog_manager.cc:1347] Generated new cluster ID: d0e867615d7846529c3043101ce27247
I20260809 03:32:45.251099 3574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:32:45.251811 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:45.244421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53252 (local address 127.27.40.190:43979)
0809 03:32:45.244665 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:45.244673 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:45.244703 (+ 30us) server_negotiation.cc:407] Connection header received
0809 03:32:45.246008 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:45.246017 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:45.246100 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:45.246290 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:45.246555 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.248360 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.249514 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.250185 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.250306 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:45.251034 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:45.251038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:45.251038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:32:45.251259 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:45.251312 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:32:45.251336 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:45.251407 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:45.251521 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 03:32:45.251600 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":112,"threads_started":1}
I20260809 03:32:45.252713 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:32:45.266167 3574 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:45.266693 3574 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:45.272184 3574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60d4a8e90e31489d88d8cb30df329feb: Generated new TSK 0
I20260809 03:32:45.272380 3574 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:32:45.287896 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:45.284004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53268 (local address 127.27.40.190:43979)
0809 03:32:45.284056 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:45.284058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:45.284100 (+ 42us) server_negotiation.cc:407] Connection header received
0809 03:32:45.284205 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:45.284206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:45.284235 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:45.284328 (+ 93us) server_negotiation.cc:226] Negotiated authn=JWT
0809 03:32:45.284520 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.285306 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.286200 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.286510 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.286582 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:45.286733 (+ 151us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 03:32:45.287686 (+ 953us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34561/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 03:32:45.287816 (+ 130us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53268: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34561/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 03:32:45.287961 3560 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53268: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34561/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 03:32:45.289950 3594 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:32:45.283879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:43979 (local address 127.0.0.1:53268)
0809 03:32:45.284034 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:45.284048 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:45.284128 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:45.284382 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:45.284383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:45.284385 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 03:32:45.284439 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:45.284441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.285392 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.285393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:45.286121 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:45.286124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.286597 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.286598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:45.286640 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:45.286645 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 03:32:45.288046 (+ 1401us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34561/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 03:32:45.289851 (+ 1805us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.40.190:43979: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34561/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":1608,"thread_start_us":86,"threads_started":1}
I20260809 03:32:45.290125 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 3451
E20260809 03:32:45.306785 3595 webserver.cc:533] Webserver: sslize failed:
E20260809 03:32:45.306890 3595 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T03:32:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (229 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T03:32:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:45Z Disabled control of system clock
I20260809 03:32:45.346182 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:40321
--webserver_interface=127.27.40.190
--webserver_port=0
--builtin_ntp_servers=127.27.40.148:43133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:40321
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/testchainca.cert with env {}
W20260809 03:32:45.429868 3602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:45.430089 3602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:45.430112 3602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:45.431210 3602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:45.431244 3602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:45.431257 3602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:45.431269 3602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:45.431282 3602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 03:32:45.431294 3602 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 03:32:45.431305 3602 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 03:32:45.432694 3602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:43133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:40321
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:40321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:45.432889 3602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:45.433048 3602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:45.437793 3608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:45.437844 3612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:45.437839 3607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:45.438114 3602 server_base.cc:1034] running on GCE node
I20260809 03:32:45.438396 3602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:45.438757 3602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:45.439939 3602 hybrid_clock.cc:648] HybridClock initialized: now 1786246365439923 us; error 27 us; skew 500 ppm
I20260809 03:32:45.441229 3602 webserver.cc:460] Webserver started at http://127.27.40.190:46311/ using document root <none> and password file <none>
I20260809 03:32:45.441514 3602 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:45.441648 3602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:45.441828 3602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:45.442939 3602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "f4ec0b103f5a49fa8d23cbc00c03a0fe"
format_stamp: "Formatted at 2026-08-09 03:32:45 on dist-test-slave-gkw7"
server_key: "3b70bc54c7f050d44255ae596a5c5147"
server_key_iv: "cfb6ec967d58987cf86229bf2252be9b"
server_key_version: "encryptionkey@0"
I20260809 03:32:45.443331 3602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "f4ec0b103f5a49fa8d23cbc00c03a0fe"
format_stamp: "Formatted at 2026-08-09 03:32:45 on dist-test-slave-gkw7"
server_key: "3b70bc54c7f050d44255ae596a5c5147"
server_key_iv: "cfb6ec967d58987cf86229bf2252be9b"
server_key_version: "encryptionkey@0"
I20260809 03:32:45.444674 3602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:32:45.445379 3619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:45.445601 3602 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:32:45.445705 3602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "f4ec0b103f5a49fa8d23cbc00c03a0fe"
format_stamp: "Formatted at 2026-08-09 03:32:45 on dist-test-slave-gkw7"
server_key: "3b70bc54c7f050d44255ae596a5c5147"
server_key_iv: "cfb6ec967d58987cf86229bf2252be9b"
server_key_version: "encryptionkey@0"
I20260809 03:32:45.445789 3602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:45.456986 3602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:45.457273 3602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:45.457420 3602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:45.461485 3602 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:40321
I20260809 03:32:45.461522 3699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:40321 every 8 connection(s)
I20260809 03:32:45.461970 3602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:45.463018 3700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:45.465445 3700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe: Bootstrap starting.
I20260809 03:32:45.466121 3700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:45.466377 3700 log.cc:824] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:45.467130 3700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe: No bootstrap required, opened a new log
I20260809 03:32:45.468842 3700 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ec0b103f5a49fa8d23cbc00c03a0fe" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40321 } }
I20260809 03:32:45.468982 3700 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:45.469007 3700 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4ec0b103f5a49fa8d23cbc00c03a0fe, State: Initialized, Role: FOLLOWER
I20260809 03:32:45.469162 3700 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ec0b103f5a49fa8d23cbc00c03a0fe" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40321 } }
I20260809 03:32:45.469256 3700 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:45.469282 3700 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:45.469313 3700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:45.469874 3700 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ec0b103f5a49fa8d23cbc00c03a0fe" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40321 } }
I20260809 03:32:45.470016 3700 leader_election.cc:302] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4ec0b103f5a49fa8d23cbc00c03a0fe; no voters:
I20260809 03:32:45.470194 3700 leader_election.cc:288] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:45.470269 3703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:45.470422 3703 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [term 1 LEADER]: Becoming Leader. State: Replica: f4ec0b103f5a49fa8d23cbc00c03a0fe, State: Running, Role: LEADER
I20260809 03:32:45.470485 3700 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:45.470531 3703 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ec0b103f5a49fa8d23cbc00c03a0fe" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40321 } }
I20260809 03:32:45.470644 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 3602
I20260809 03:32:45.470777 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:45.470839 3703 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4ec0b103f5a49fa8d23cbc00c03a0fe. Latest consensus state: current_term: 1 leader_uuid: "f4ec0b103f5a49fa8d23cbc00c03a0fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ec0b103f5a49fa8d23cbc00c03a0fe" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40321 } } }
I20260809 03:32:45.470844 3704 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4ec0b103f5a49fa8d23cbc00c03a0fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ec0b103f5a49fa8d23cbc00c03a0fe" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40321 } } }
I20260809 03:32:45.470934 3703 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:45.470964 3704 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:45.470997 27810 external_mini_cluster.cc:1428] Setting key 115a967eedda7afe687f847340767b6d
I20260809 03:32:45.471284 3710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:45.471813 3710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:45.473775 3710 catalog_manager.cc:1347] Generated new cluster ID: 823f2995cd71446ba9ed582d83aae0f9
I20260809 03:32:45.473829 3710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:32:45.478592 3721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:45.471499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57432 (local address 127.27.40.190:40321)
0809 03:32:45.471786 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:45.471797 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:45.472734 (+ 937us) server_negotiation.cc:407] Connection header received
0809 03:32:45.472901 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:45.472907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:45.472959 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:45.473122 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:45.473482 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.475079 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.476177 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.476934 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.477074 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:45.477823 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:45.477827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:45.477828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:32:45.478054 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:45.478100 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:32:45.478117 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:45.478188 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:45.478263 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 03:32:45.478347 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":84,"threads_started":1}
I20260809 03:32:45.479647 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:32:45.483397 3710 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:45.484130 3710 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:45.490866 3710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4ec0b103f5a49fa8d23cbc00c03a0fe: Generated new TSK 0
I20260809 03:32:45.491065 3710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:32:45.520084 3721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:45.499377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57438 (local address 127.27.40.190:40321)
0809 03:32:45.499440 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:45.499443 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:45.499595 (+ 152us) server_negotiation.cc:407] Connection header received
0809 03:32:45.499671 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:45.499672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:45.499692 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:45.499755 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0809 03:32:45.499904 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.500410 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.501603 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.501856 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.501920 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:45.502112 (+ 192us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 03:32:45.519872 (+ 17760us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 03:32:45.520024 (+ 152us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57438: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 03:32:45.520176 3721 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57438: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 03:32:45.520371 3744 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:32:45.499312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40321 (local address 127.0.0.1:57438)
0809 03:32:45.499512 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:45.499524 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:45.499600 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:45.499772 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:45.499773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:45.499775 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 03:32:45.499838 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:45.499840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.500505 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.500506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:45.501469 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:45.501472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:45.501947 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:45.501949 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:45.502014 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:45.502020 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 03:32:45.520014 (+ 17994us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 03:32:45.520157 (+ 143us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.40.190:40321: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":131,"threads_started":1}
I20260809 03:32:45.520743 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 3602
2026-08-09T03:32:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (204 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (93 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (91 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:32:45 dist-test-slave-gkw7 krb5kdc[3760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:32:45 dist-test-slave-gkw7 krb5kdc[3760](info): set up 2 sockets
Aug 09 03:32:45 dist-test-slave-gkw7 krb5kdc[3760](info): commencing operation
krb5kdc: starting...
W20260809 03:32:46.955212 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:32:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:32:47.088299 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:42825
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:35935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:42825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:47.172974 3778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:47.173192 3778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:47.173215 3778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:47.174413 3778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:47.174453 3778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:47.174466 3778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:47.174479 3778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:47.174490 3778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:32:47.175906 3778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:35935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:42825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:42825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:47.176106 3778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:47.176280 3778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:47.181197 3787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:47.181223 3790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:47.181412 3778 server_base.cc:1034] running on GCE node
W20260809 03:32:47.181229 3788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:47.181720 3778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:47.182029 3778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:47.183171 3778 hybrid_clock.cc:648] HybridClock initialized: now 1786246367183159 us; error 27 us; skew 500 ppm
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:47.185810 3778 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:32:47.186288 3778 webserver.cc:460] Webserver started at http://127.27.40.190:39993/ using document root <none> and password file <none>
I20260809 03:32:47.186529 3778 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:47.186600 3778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:47.186756 3778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:47.187781 3778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "f43f285a527243989d67f1ec4514cb04"
format_stamp: "Formatted at 2026-08-09 03:32:47 on dist-test-slave-gkw7"
server_key: "3a545b57d752da39ef21add6a55e45cc"
server_key_iv: "adad5c9f685beb6959c259e945342fd1"
server_key_version: "encryptionkey@0"
I20260809 03:32:47.188144 3778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "f43f285a527243989d67f1ec4514cb04"
format_stamp: "Formatted at 2026-08-09 03:32:47 on dist-test-slave-gkw7"
server_key: "3a545b57d752da39ef21add6a55e45cc"
server_key_iv: "adad5c9f685beb6959c259e945342fd1"
server_key_version: "encryptionkey@0"
I20260809 03:32:47.189510 3778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:32:47.190380 3805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:47.190567 3778 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:47.190670 3778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "f43f285a527243989d67f1ec4514cb04"
format_stamp: "Formatted at 2026-08-09 03:32:47 on dist-test-slave-gkw7"
server_key: "3a545b57d752da39ef21add6a55e45cc"
server_key_iv: "adad5c9f685beb6959c259e945342fd1"
server_key_version: "encryptionkey@0"
I20260809 03:32:47.190760 3778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:47.204509 3778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:47.206465 3778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:47.206630 3778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:47.210831 3778 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:42825
I20260809 03:32:47.210872 3876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:42825 every 8 connection(s)
I20260809 03:32:47.211251 3778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:47.212169 3877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:47.214072 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 3778
I20260809 03:32:47.214183 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:47.214396 27810 external_mini_cluster.cc:1428] Setting key 107e717dfd78f013c50b87fc8f746fe6
I20260809 03:32:47.214676 3877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04: Bootstrap starting.
I20260809 03:32:47.215442 3877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:47.215783 3877 log.cc:824] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:47.216550 3877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04: No bootstrap required, opened a new log
I20260809 03:32:47.218855 3877 raft_consensus.cc:348] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43f285a527243989d67f1ec4514cb04" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 42825 } }
I20260809 03:32:47.219043 3877 raft_consensus.cc:374] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:47.219079 3877 raft_consensus.cc:729] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f43f285a527243989d67f1ec4514cb04, State: Initialized, Role: FOLLOWER
I20260809 03:32:47.219211 3877 consensus_queue.cc:260] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43f285a527243989d67f1ec4514cb04" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 42825 } }
I20260809 03:32:47.219282 3877 raft_consensus.cc:388] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:47.219312 3877 raft_consensus.cc:482] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:47.219357 3877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:47.219995 3877 raft_consensus.cc:504] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43f285a527243989d67f1ec4514cb04" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 42825 } }
I20260809 03:32:47.220124 3877 leader_election.cc:302] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f43f285a527243989d67f1ec4514cb04; no voters:
I20260809 03:32:47.220314 3877 leader_election.cc:288] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:47.220368 3884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:47.220542 3884 raft_consensus.cc:686] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [term 1 LEADER]: Becoming Leader. State: Replica: f43f285a527243989d67f1ec4514cb04, State: Running, Role: LEADER
I20260809 03:32:47.220575 3877 sys_catalog.cc:564] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:47.220690 3884 consensus_queue.cc:237] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43f285a527243989d67f1ec4514cb04" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 42825 } }
I20260809 03:32:47.221112 3886 sys_catalog.cc:455] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f43f285a527243989d67f1ec4514cb04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43f285a527243989d67f1ec4514cb04" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 42825 } } }
I20260809 03:32:47.221195 3886 sys_catalog.cc:458] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:47.221414 3889 sys_catalog.cc:455] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f43f285a527243989d67f1ec4514cb04. Latest consensus state: current_term: 1 leader_uuid: "f43f285a527243989d67f1ec4514cb04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43f285a527243989d67f1ec4514cb04" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 42825 } } }
I20260809 03:32:47.221737 3894 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:47.221746 3889 sys_catalog.cc:458] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04 [sys.catalog]: This master's current role is: LEADER
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:47.222298 3894 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:47.224484 3894 catalog_manager.cc:1347] Generated new cluster ID: d101607864984085958f5d011673c485
I20260809 03:32:47.224539 3894 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:32:47.226389 3881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:47.214949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.27.40.190:42825)
0809 03:32:47.215160 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:47.215170 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:47.215205 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:32:47.215868 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:47.215877 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:47.215939 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:47.216085 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:47.216763 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.218184 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.218941 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.219556 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.219680 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:47.222890 (+ 3210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:47.222895 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:47.222896 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:47.222908 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:47.224843 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:47.225247 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:47.225250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:47.225250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:47.225344 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:47.225590 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:47.225591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:47.225592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:47.225825 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:47.225988 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:47.226088 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 03:32:47.226245 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":96,"threads_started":1}
I20260809 03:32:47.231932 3894 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:47.232422 3894 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:47.235344 3894 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f43f285a527243989d67f1ec4514cb04: Generated new TSK 0
I20260809 03:32:47.235517 3894 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:32:47.264303 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:42825
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:35935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:47.352749 3916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:47.352952 3916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:47.352975 3916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:47.354184 3916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:47.354233 3916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:47.354260 3916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:32:47.355729 3916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:35935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:42825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:47.355975 3916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:47.356226 3916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:47.361227 3923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:47.361282 3927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:47.361263 3924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:47.361601 3916 server_base.cc:1034] running on GCE node
I20260809 03:32:47.361827 3916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:47.362073 3916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:47.363250 3916 hybrid_clock.cc:648] HybridClock initialized: now 1786246367363241 us; error 27 us; skew 500 ppm
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:47.365794 3916 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:32:47.366264 3916 webserver.cc:460] Webserver started at http://127.27.40.129:32801/ using document root <none> and password file <none>
I20260809 03:32:47.366508 3916 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:47.366576 3916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:47.366730 3916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:47.367750 3916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "2cfa90f32fd64fb3b69f96a75a9971d4"
format_stamp: "Formatted at 2026-08-09 03:32:47 on dist-test-slave-gkw7"
server_key: "4d0f66cb08ce24c6f4efbf303bdd835d"
server_key_iv: "5f495f37f696290659e8644e85545f58"
server_key_version: "encryptionkey@0"
I20260809 03:32:47.368106 3916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "2cfa90f32fd64fb3b69f96a75a9971d4"
format_stamp: "Formatted at 2026-08-09 03:32:47 on dist-test-slave-gkw7"
server_key: "4d0f66cb08ce24c6f4efbf303bdd835d"
server_key_iv: "5f495f37f696290659e8644e85545f58"
server_key_version: "encryptionkey@0"
I20260809 03:32:47.369414 3916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 03:32:47.370203 3937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:47.370373 3916 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:32:47.370469 3916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "2cfa90f32fd64fb3b69f96a75a9971d4"
format_stamp: "Formatted at 2026-08-09 03:32:47 on dist-test-slave-gkw7"
server_key: "4d0f66cb08ce24c6f4efbf303bdd835d"
server_key_iv: "5f495f37f696290659e8644e85545f58"
server_key_version: "encryptionkey@0"
I20260809 03:32:47.370569 3916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:47.386449 3916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:47.388887 3916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:47.389062 3916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:47.389321 3916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:47.389721 3916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:47.389773 3916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:47.389804 3916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:47.389822 3916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:47.396996 3916 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:35045
I20260809 03:32:47.397143 4101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:35045 every 8 connection(s)
I20260809 03:32:47.397444 3916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:32:47.400477 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 3916
I20260809 03:32:47.400588 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:32:47.400805 27810 external_mini_cluster.cc:1428] Setting key 67254ce122e40eecdec5951a11f7a977
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:47.408612 3881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:47.398242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:54599 (local address 127.27.40.190:42825)
0809 03:32:47.398314 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:47.398316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:47.398532 (+ 216us) server_negotiation.cc:407] Connection header received
0809 03:32:47.399579 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:47.399581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:47.399591 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:47.399635 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:47.400401 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.401212 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.402430 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.402757 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.402834 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:47.405321 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:47.405322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:47.405323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:47.405331 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:47.406973 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:47.407468 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:47.407469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:47.407470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:47.407542 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:47.407852 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:47.407853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:47.407853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:47.407988 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:47.408071 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:47.408397 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 03:32:47.408506 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 03:32:47.408867 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:47.398225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:42825 (local address 127.27.40.129:54599)
0809 03:32:47.398402 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:47.398441 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:47.399469 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:47.399695 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:47.399698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:47.399984 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:47.400306 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:47.400313 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.401312 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.401313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:47.402339 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:47.402343 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.402856 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.402857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:47.402986 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:47.403653 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:47.403662 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:47.405195 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:47.407090 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:47.407100 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:47.407103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:47.407367 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:47.407627 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:47.407628 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:47.407628 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:47.407764 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:47.408075 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:47.408084 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:47.408246 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 03:32:47.408373 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 03:32:47.408560 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
I20260809 03:32:47.409608 4103 heartbeater.cc:347] Connected to a master server at 127.27.40.190:42825
I20260809 03:32:47.409719 4103 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:47.409883 4103 heartbeater.cc:511] Master 127.27.40.190:42825 requested a full tablet report, sending...
I20260809 03:32:47.410470 3830 ts_manager.cc:194] Registered new tserver with Master: 2cfa90f32fd64fb3b69f96a75a9971d4 (127.27.40.129:35045)
I20260809 03:32:47.411113 3830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:54599
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:32:47.428215 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:42825
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:35935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:47.511868 4111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:47.512069 4111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:47.512091 4111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:47.513259 4111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:47.513320 4111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:47.513348 4111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:32:47.514889 4111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:35935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:42825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:47.515126 4111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:47.515383 4111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:47.520589 4123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:47.520591 4120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:47.520588 4117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:47.520722 4111 server_base.cc:1034] running on GCE node
I20260809 03:32:47.521023 4111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:47.521274 4111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:47.522466 4111 hybrid_clock.cc:648] HybridClock initialized: now 1786246367522457 us; error 34 us; skew 500 ppm
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:47.524981 4111 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:32:47.525406 4111 webserver.cc:460] Webserver started at http://127.27.40.130:37465/ using document root <none> and password file <none>
I20260809 03:32:47.525717 4111 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:47.525785 4111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:47.525954 4111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:47.527041 4111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "6148f901d3024be2abca002a7069012b"
format_stamp: "Formatted at 2026-08-09 03:32:47 on dist-test-slave-gkw7"
server_key: "134de22867df348268492d856b400a60"
server_key_iv: "a3f7a3def45023d5f9c505473ad15e80"
server_key_version: "encryptionkey@0"
I20260809 03:32:47.527475 4111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "6148f901d3024be2abca002a7069012b"
format_stamp: "Formatted at 2026-08-09 03:32:47 on dist-test-slave-gkw7"
server_key: "134de22867df348268492d856b400a60"
server_key_iv: "a3f7a3def45023d5f9c505473ad15e80"
server_key_version: "encryptionkey@0"
I20260809 03:32:47.528832 4111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:32:47.529522 4131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:47.529738 4111 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:47.529836 4111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "6148f901d3024be2abca002a7069012b"
format_stamp: "Formatted at 2026-08-09 03:32:47 on dist-test-slave-gkw7"
server_key: "134de22867df348268492d856b400a60"
server_key_iv: "a3f7a3def45023d5f9c505473ad15e80"
server_key_version: "encryptionkey@0"
I20260809 03:32:47.529918 4111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:47.562325 4111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:47.564455 4111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:47.564640 4111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:47.564925 4111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:47.565300 4111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:47.565362 4111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:47.565413 4111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:47.565449 4111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:47.572119 4111 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:38803
I20260809 03:32:47.572165 4301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:38803 every 8 connection(s)
I20260809 03:32:47.572563 4111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:32:47.574259 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 4111
I20260809 03:32:47.574359 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:32:47.574550 27810 external_mini_cluster.cc:1428] Setting key 3967c8024df51ea8426307af416a204a
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:47.583212 3881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:47.573437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:50803 (local address 127.27.40.190:42825)
0809 03:32:47.573562 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:47.573564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:47.573742 (+ 178us) server_negotiation.cc:407] Connection header received
0809 03:32:47.574961 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:47.574962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:47.574976 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:47.575021 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:47.575760 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.576524 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.577520 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.577893 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.577971 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:47.580243 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:47.580244 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:47.580245 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:47.580252 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:47.581789 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:47.582221 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:47.582222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:47.582222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:47.582287 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:47.582541 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:47.582542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:47.582542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:47.582671 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:47.582749 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:47.583022 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 03:32:47.583110 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 03:32:47.583441 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:47.573390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:42825 (local address 127.27.40.130:50803)
0809 03:32:47.573604 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:47.573653 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:47.574830 (+ 1177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:47.575089 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:47.575092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:47.575464 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:47.575666 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:47.575671 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.576616 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.576617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:47.577432 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:47.577434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.577985 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.577986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:47.578096 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:47.578678 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:47.578686 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:47.580130 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:47.581901 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:47.581908 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:47.581910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:47.582142 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:47.582358 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:47.582359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:47.582359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:47.582472 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:47.582740 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:47.582747 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:47.582890 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 03:32:47.583002 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 03:32:47.583199 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":110,"threads_started":1}
I20260809 03:32:47.583932 4303 heartbeater.cc:347] Connected to a master server at 127.27.40.190:42825
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
I20260809 03:32:47.584029 4303 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:47.584241 4303 heartbeater.cc:511] Master 127.27.40.190:42825 requested a full tablet report, sending...
I20260809 03:32:47.584640 3830 ts_manager.cc:194] Registered new tserver with Master: 6148f901d3024be2abca002a7069012b (127.27.40.130:38803)
I20260809 03:32:47.585064 3830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:50803
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:32:47.601984 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:42825
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:35935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:47.684737 4309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:47.684994 4309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:47.685056 4309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:47.686224 4309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:47.686306 4309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:47.686352 4309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:32:47.687850 4309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:35935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:42825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:47.688156 4309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:47.688395 4309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:47.693464 4321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:47.693462 4316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:47.693760 4309 server_base.cc:1034] running on GCE node
W20260809 03:32:47.693531 4319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:47.694294 4309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:47.694607 4309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:47.695775 4309 hybrid_clock.cc:648] HybridClock initialized: now 1786246367695762 us; error 32 us; skew 500 ppm
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:47.698429 4309 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:32:47.698904 4309 webserver.cc:460] Webserver started at http://127.27.40.131:33513/ using document root <none> and password file <none>
I20260809 03:32:47.699152 4309 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:47.699221 4309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:47.699368 4309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:47.700382 4309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "daf127055d58484dbca115429e9017d6"
format_stamp: "Formatted at 2026-08-09 03:32:47 on dist-test-slave-gkw7"
server_key: "6ac9107eceb93956a497288001c5f18b"
server_key_iv: "8f0edd952889bec1f07a40b9927b8dc4"
server_key_version: "encryptionkey@0"
I20260809 03:32:47.700816 4309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "daf127055d58484dbca115429e9017d6"
format_stamp: "Formatted at 2026-08-09 03:32:47 on dist-test-slave-gkw7"
server_key: "6ac9107eceb93956a497288001c5f18b"
server_key_iv: "8f0edd952889bec1f07a40b9927b8dc4"
server_key_version: "encryptionkey@0"
I20260809 03:32:47.702332 4309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:32:47.703059 4331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:47.703239 4309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:32:47.703339 4309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "daf127055d58484dbca115429e9017d6"
format_stamp: "Formatted at 2026-08-09 03:32:47 on dist-test-slave-gkw7"
server_key: "6ac9107eceb93956a497288001c5f18b"
server_key_iv: "8f0edd952889bec1f07a40b9927b8dc4"
server_key_version: "encryptionkey@0"
I20260809 03:32:47.703415 4309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:47.737983 4309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:47.740418 4309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:47.740612 4309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:47.740854 4309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:47.741174 4309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:47.741227 4309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:47.741269 4309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:47.741294 4309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:47.748193 4309 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:43515
I20260809 03:32:47.748237 4507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:43515 every 8 connection(s)
I20260809 03:32:47.748569 4309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:32:47.757503 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 4309
I20260809 03:32:47.757644 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:32:47.757835 27810 external_mini_cluster.cc:1428] Setting key 40e33a54e493137c8ebd02aa2befdba1
I20260809 03:32:47.757951 3881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:47.749368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:32819 (local address 127.27.40.190:42825)
0809 03:32:47.749415 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:47.749417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:47.749615 (+ 198us) server_negotiation.cc:407] Connection header received
0809 03:32:47.750680 (+ 1065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:47.750681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:47.750692 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:47.750762 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:47.751420 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.751920 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.752905 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.753090 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.753144 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:47.755390 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:47.755392 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:47.755392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:47.755400 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:47.756598 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:47.756961 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:47.756962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:47.756962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:47.757015 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:47.757231 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:47.757231 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:47.757231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:47.757381 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:47.757457 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:47.757769 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 03:32:47.757863 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 03:32:47.758167 4510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:47.749307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:42825 (local address 127.27.40.131:32819)
0809 03:32:47.749470 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:47.749513 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:47.750588 (+ 1075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:47.750821 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:47.750824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:47.751127 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:47.751340 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:47.751344 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.752002 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.752003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:47.752826 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:47.752828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.753182 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.753183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:47.753293 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:47.753877 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:47.753886 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:47.755299 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:47.756674 (+ 1375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:47.756680 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:47.756681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:47.756895 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:47.757071 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:47.757072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:47.757072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:47.757177 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:47.757448 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:47.757452 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:47.757618 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 03:32:47.757743 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 03:32:47.757915 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260809 03:32:47.758646 4508 heartbeater.cc:347] Connected to a master server at 127.27.40.190:42825
I20260809 03:32:47.758751 4508 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:47.758949 4508 heartbeater.cc:511] Master 127.27.40.190:42825 requested a full tablet report, sending...
I20260809 03:32:47.759343 3829 ts_manager.cc:194] Registered new tserver with Master: daf127055d58484dbca115429e9017d6 (127.27.40.131:43515)
I20260809 03:32:47.759681 3829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:32819
I20260809 03:32:47.769683 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/test-admin.keytab.
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:47.786402 27810 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 03:32:47.788179 4515 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:32:47.787350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35638 (local address 127.27.40.129:35045)
0809 03:32:47.787564 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:47.787569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:47.787579 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:32:47.787605 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:47.787607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:47.787623 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:47.787700 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:47.788076 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35638: BlockingRecv error: recv got EOF from 127.0.0.1:35638 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":106,"threads_started":1}
W20260809 03:32:47.788349 27810 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/krb5cc' of type FILE
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:47.789325 27810 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:47.790315 27810 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.129@KRBTEST.COM
I20260809 03:32:47.801992 4515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:47.790833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35652 (local address 127.27.40.129:35045)
0809 03:32:47.790912 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:47.790913 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:47.790965 (+ 52us) server_negotiation.cc:407] Connection header received
0809 03:32:47.791004 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:47.791004 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:47.791014 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:47.791062 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:47.791440 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.792165 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.792885 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.793194 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.793266 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:47.799349 (+ 6083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:47.799358 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:47.799359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:47.799366 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:47.800822 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:47.801233 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:47.801236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:47.801236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:47.801306 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:47.801476 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:47.801477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:47.801477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:47.801657 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:47.801762 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:47.801849 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 03:32:47.801915 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 03:32:47.803910 4515 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:32:47.803355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35666 (local address 127.27.40.129:35045)
0809 03:32:47.803408 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:47.803410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:47.803456 (+ 46us) server_negotiation.cc:407] Connection header received
0809 03:32:47.803503 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:47.803504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:47.803513 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:47.803562 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:47.803864 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35666: BlockingRecv error: recv got EOF from 127.0.0.1:35666 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 03:32:47.804016 27810 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/krb5cc' of type FILE
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:47.805014 27810 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:47.806048 27810 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.129@KRBTEST.COM
I20260809 03:32:47.813208 4515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:47.806579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.27.40.129:35045)
0809 03:32:47.806656 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:47.806658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:47.806697 (+ 39us) server_negotiation.cc:407] Connection header received
0809 03:32:47.806749 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:47.806750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:47.806758 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:47.806818 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:47.807241 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.807752 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.808409 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.808614 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.808675 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:47.810929 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:47.810931 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:47.810931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:47.810939 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:47.812329 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:47.812646 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:47.812647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:47.812647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:47.812698 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:47.812859 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:47.812859 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:47.812859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:47.812939 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:47.813008 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:47.813090 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 03:32:47.813148 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 03:32:47.814725 4515 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:32:47.814221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35686 (local address 127.27.40.129:35045)
0809 03:32:47.814259 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:47.814260 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:47.814351 (+ 91us) server_negotiation.cc:407] Connection header received
0809 03:32:47.814384 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:47.814385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:47.814393 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:47.814435 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:47.814656 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35686: BlockingRecv error: recv got EOF from 127.0.0.1:35686 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 03:32:47.814810 27810 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246339702443-27810-0/krb5kdc/krb5cc' of type FILE
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:47.815732 27810 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:47.816543 27810 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.129@KRBTEST.COM
I20260809 03:32:47.823545 4515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:47.817062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.27.40.129:35045)
0809 03:32:47.817099 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:47.817100 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:47.817156 (+ 56us) server_negotiation.cc:407] Connection header received
0809 03:32:47.817204 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:47.817204 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:47.817211 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:47.817253 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:47.817614 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.818086 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.818761 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:47.818958 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:47.819007 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:47.821062 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:47.821063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:47.821064 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:47.821075 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:47.822329 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:47.822788 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:47.822790 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:47.822790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:47.822847 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:47.823053 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:47.823054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:47.823055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:47.823153 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:47.823263 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:47.823380 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 03:32:47.823451 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 03:32:47.824177 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 3916
I20260809 03:32:47.837728 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 4111
I20260809 03:32:47.845234 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 4309
I20260809 03:32:47.853356 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 3778
2026-08-09T03:32:47Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2152 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[4540](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[4540](info): set up 2 sockets
Aug 09 03:32:47 dist-test-slave-gkw7 krb5kdc[4540](info): commencing operation
krb5kdc: starting...
W20260809 03:32:49.076959 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:32:49 dist-test-slave-gkw7 krb5kdc[4540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:32:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:49Z Disabled control of system clock
WARNING: no policy specified for oryx/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.190.keytab.
Entry for principal oryx/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.190.keytab.
I20260809 03:32:49.209347 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:40413
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.190.keytab
--principal=oryx/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:40413
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:49.295277 4563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:49.295493 4563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:49.295516 4563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:49.296690 4563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:49.296736 4563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:49.296751 4563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:49.296763 4563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:49.296775 4563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:32:49.298456 4563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:40413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.190.keytab
--principal=oryx/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:40413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:49.298738 4563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:49.298988 4563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:49.304059 4571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:49.304230 4563 server_base.cc:1034] running on GCE node
W20260809 03:32:49.304037 4570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:49.304064 4573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:49.304657 4563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:49.304977 4563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:49.306123 4563 hybrid_clock.cc:648] HybridClock initialized: now 1786246369306104 us; error 40 us; skew 500 ppm
Aug 09 03:32:49 dist-test-slave-gkw7 krb5kdc[4540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246369, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:49.308784 4563 init.cc:377] Logged in from keytab as oryx/127.27.40.190@KRBTEST.COM (short username oryx)
I20260809 03:32:49.309243 4563 webserver.cc:460] Webserver started at http://127.27.40.190:43869/ using document root <none> and password file <none>
I20260809 03:32:49.309494 4563 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:49.309597 4563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:49.309749 4563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:49.310842 4563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "0b478a01c8d24855948ebf23178e3c8f"
format_stamp: "Formatted at 2026-08-09 03:32:49 on dist-test-slave-gkw7"
server_key: "980aa5606eba83c04fa7fc590db68fca"
server_key_iv: "a2c506357a6d9f925c03bec82b354695"
server_key_version: "encryptionkey@0"
I20260809 03:32:49.311218 4563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "0b478a01c8d24855948ebf23178e3c8f"
format_stamp: "Formatted at 2026-08-09 03:32:49 on dist-test-slave-gkw7"
server_key: "980aa5606eba83c04fa7fc590db68fca"
server_key_iv: "a2c506357a6d9f925c03bec82b354695"
server_key_version: "encryptionkey@0"
I20260809 03:32:49.312625 4563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:32:49.313369 4583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:49.313601 4563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:32:49.313705 4563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "0b478a01c8d24855948ebf23178e3c8f"
format_stamp: "Formatted at 2026-08-09 03:32:49 on dist-test-slave-gkw7"
server_key: "980aa5606eba83c04fa7fc590db68fca"
server_key_iv: "a2c506357a6d9f925c03bec82b354695"
server_key_version: "encryptionkey@0"
I20260809 03:32:49.313792 4563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:49.333341 4563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:49.335588 4563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:49.335757 4563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:49.339900 4563 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:40413
I20260809 03:32:49.339937 4665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:40413 every 8 connection(s)
I20260809 03:32:49.340328 4563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:49.341456 4666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:49.343878 4666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f: Bootstrap starting.
I20260809 03:32:49.344496 4666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:49.344791 4666 log.cc:824] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:49.345500 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 4563
I20260809 03:32:49.345510 4666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f: No bootstrap required, opened a new log
I20260809 03:32:49.345666 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:49.345882 27810 external_mini_cluster.cc:1428] Setting key b2208f4a4490a9ea658dd673279ca5e0
I20260809 03:32:49.347929 4666 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } }
I20260809 03:32:49.348107 4666 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:49.348158 4666 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b478a01c8d24855948ebf23178e3c8f, State: Initialized, Role: FOLLOWER
I20260809 03:32:49.348316 4666 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } }
I20260809 03:32:49.348383 4666 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:49.348404 4666 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:49.348469 4666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:49.349514 4666 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } }
I20260809 03:32:49.349642 4666 leader_election.cc:302] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b478a01c8d24855948ebf23178e3c8f; no voters:
I20260809 03:32:49.349804 4666 leader_election.cc:288] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:49.349864 4675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:49.350036 4666 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:49.350049 4675 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [term 1 LEADER]: Becoming Leader. State: Replica: 0b478a01c8d24855948ebf23178e3c8f, State: Running, Role: LEADER
I20260809 03:32:49.350339 4675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } }
I20260809 03:32:49.350726 4676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } }
I20260809 03:32:49.350807 4676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:49.350988 4677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b478a01c8d24855948ebf23178e3c8f. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } }
I20260809 03:32:49.351084 4677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:49.351058 4688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:49.351792 4688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:49.353382 4688 catalog_manager.cc:1347] Generated new cluster ID: e78f5199826845eeb37908ef76813cbe
I20260809 03:32:49.353439 4688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 03:32:49 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM
I20260809 03:32:49.358347 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:49.346544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.27.40.190:40413)
0809 03:32:49.346767 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:49.346776 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:49.346810 (+ 34us) server_negotiation.cc:407] Connection header received
0809 03:32:49.347893 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:49.347904 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:49.347970 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:49.348126 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:49.348972 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:49.350417 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:49.351554 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:49.352164 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:49.352292 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:49.354836 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:49.354840 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:49.354840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:49.354852 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:49.356562 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:49.356939 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:49.356942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:49.356943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:49.357052 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:49.357283 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:49.357285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:49.357285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:49.357571 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:49.357752 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:49.357846 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 03:32:49.357996 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":101,"threads_started":1}
I20260809 03:32:49.362483 4688 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:49.363168 4688 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:49.384393 4688 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f: Generated new TSK 0
I20260809 03:32:49.384636 4688 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.129.keytab.
Entry for principal oryx/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.129.keytab.
I20260809 03:32:49.414624 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40413
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.129.keytab
--principal=oryx/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:49.498163 4709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:49.498373 4709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:49.498395 4709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:49.499617 4709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:49.499660 4709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:49.499707 4709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:32:49.501299 4709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.129.keytab
--principal=oryx/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:49.501585 4709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:49.501833 4709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:49.507187 4719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:49.507239 4717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:49.507156 4716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:49.507272 4709 server_base.cc:1034] running on GCE node
I20260809 03:32:49.507472 4709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:49.507754 4709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:49.508934 4709 hybrid_clock.cc:648] HybridClock initialized: now 1786246369508922 us; error 31 us; skew 500 ppm
Aug 09 03:32:49 dist-test-slave-gkw7 krb5kdc[4540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246369, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:49.511730 4709 init.cc:377] Logged in from keytab as oryx/127.27.40.129@KRBTEST.COM (short username oryx)
I20260809 03:32:49.512123 4709 webserver.cc:460] Webserver started at http://127.27.40.129:36057/ using document root <none> and password file <none>
I20260809 03:32:49.512333 4709 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:49.512374 4709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:49.512463 4709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:49.513468 4709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "6694ec3787544426b332b9535dbe9ff3"
format_stamp: "Formatted at 2026-08-09 03:32:49 on dist-test-slave-gkw7"
server_key: "4ee146ea83d839a30710409fb7707267"
server_key_iv: "a95d872f62718cfa0542c36d5568dede"
server_key_version: "encryptionkey@0"
I20260809 03:32:49.513870 4709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "6694ec3787544426b332b9535dbe9ff3"
format_stamp: "Formatted at 2026-08-09 03:32:49 on dist-test-slave-gkw7"
server_key: "4ee146ea83d839a30710409fb7707267"
server_key_iv: "a95d872f62718cfa0542c36d5568dede"
server_key_version: "encryptionkey@0"
I20260809 03:32:49.515177 4709 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:49.515877 4733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:49.516037 4709 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:49.516126 4709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "6694ec3787544426b332b9535dbe9ff3"
format_stamp: "Formatted at 2026-08-09 03:32:49 on dist-test-slave-gkw7"
server_key: "4ee146ea83d839a30710409fb7707267"
server_key_iv: "a95d872f62718cfa0542c36d5568dede"
server_key_version: "encryptionkey@0"
I20260809 03:32:49.516184 4709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:49.533732 4709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:49.536116 4709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:49.536244 4709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:49.536434 4709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:49.536788 4709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:49.536826 4709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:49.536849 4709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:49.536862 4709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:49.542883 4709 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:39613
I20260809 03:32:49.542991 4893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:39613 every 8 connection(s)
I20260809 03:32:49.543308 4709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
Aug 09 03:32:49 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246369, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM
I20260809 03:32:49.550832 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 4709
I20260809 03:32:49.550957 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:32:49.551255 27810 external_mini_cluster.cc:1428] Setting key 64cb6cc0a9f213892d3a6ab59d5a584d
I20260809 03:32:49.554395 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:49.544115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:48049 (local address 127.27.40.190:40413)
0809 03:32:49.544330 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:49.544333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:49.544533 (+ 200us) server_negotiation.cc:407] Connection header received
0809 03:32:49.545174 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:49.545176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:49.545193 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:49.545261 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:49.546426 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:49.547002 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:49.548124 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:49.548399 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:49.548491 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:49.551038 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:49.551040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:49.551041 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:49.551052 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:49.552597 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:49.553154 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:49.553155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:49.553155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:49.553218 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:49.553569 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:49.553570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:49.553571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:49.553693 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:49.553751 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:49.554149 (+ 398us) server_negotiation.cc:299] Negotiation successful
0809 03:32:49.554263 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260809 03:32:49.554487 4897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:49.544026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.129:48049)
0809 03:32:49.544370 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:49.544424 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:49.545033 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:49.545337 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:49.545342 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:49.546055 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:49.546303 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:49.546308 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:49.547122 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:49.547123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:49.548019 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:49.548021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:49.548501 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:49.548502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:49.548662 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:49.549160 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:49.549170 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:49.550929 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:49.552689 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:49.552697 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:49.552700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:49.553068 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:49.553287 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:49.553288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:49.553289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:49.553458 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:49.553767 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:49.553774 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:49.553966 (+ 192us) client_negotiation.cc:769] Sending connection context
0809 03:32:49.554108 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 03:32:49.554279 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":94,"threads_started":1}
I20260809 03:32:49.555068 4894 heartbeater.cc:347] Connected to a master server at 127.27.40.190:40413
I20260809 03:32:49.555172 4894 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:49.555390 4894 heartbeater.cc:511] Master 127.27.40.190:40413 requested a full tablet report, sending...
I20260809 03:32:49.556033 4610 ts_manager.cc:194] Registered new tserver with Master: 6694ec3787544426b332b9535dbe9ff3 (127.27.40.129:39613)
I20260809 03:32:49.557101 4610 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:48049
WARNING: no policy specified for oryx/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.130.keytab.
Entry for principal oryx/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.130.keytab.
I20260809 03:32:49.581364 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40413
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.130.keytab
--principal=oryx/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:49.667348 4902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:49.667570 4902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:49.667593 4902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:49.668763 4902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:49.668810 4902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:49.668836 4902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:32:49.670326 4902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.130.keytab
--principal=oryx/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:49.670575 4902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:49.670825 4902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:49.675881 4910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:49.675890 4912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:49.675941 4908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:49.676090 4902 server_base.cc:1034] running on GCE node
I20260809 03:32:49.676292 4902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:49.676568 4902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:49.677734 4902 hybrid_clock.cc:648] HybridClock initialized: now 1786246369677729 us; error 44 us; skew 500 ppm
Aug 09 03:32:49 dist-test-slave-gkw7 krb5kdc[4540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246369, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:49.680437 4902 init.cc:377] Logged in from keytab as oryx/127.27.40.130@KRBTEST.COM (short username oryx)
I20260809 03:32:49.680958 4902 webserver.cc:460] Webserver started at http://127.27.40.130:40833/ using document root <none> and password file <none>
I20260809 03:32:49.681239 4902 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:49.681317 4902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:49.681499 4902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:49.682801 4902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "f20d19f3ea0c469f95ba10c2417139bb"
format_stamp: "Formatted at 2026-08-09 03:32:49 on dist-test-slave-gkw7"
server_key: "7a31e60b1e9ba4cf58bfde5dc0478dcb"
server_key_iv: "351dfe0aa3cb0227e2f1d7d69974ca52"
server_key_version: "encryptionkey@0"
I20260809 03:32:49.683210 4902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "f20d19f3ea0c469f95ba10c2417139bb"
format_stamp: "Formatted at 2026-08-09 03:32:49 on dist-test-slave-gkw7"
server_key: "7a31e60b1e9ba4cf58bfde5dc0478dcb"
server_key_iv: "351dfe0aa3cb0227e2f1d7d69974ca52"
server_key_version: "encryptionkey@0"
I20260809 03:32:49.684863 4902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:32:49.685719 4922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:49.685904 4902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:32:49.686017 4902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "f20d19f3ea0c469f95ba10c2417139bb"
format_stamp: "Formatted at 2026-08-09 03:32:49 on dist-test-slave-gkw7"
server_key: "7a31e60b1e9ba4cf58bfde5dc0478dcb"
server_key_iv: "351dfe0aa3cb0227e2f1d7d69974ca52"
server_key_version: "encryptionkey@0"
I20260809 03:32:49.686152 4902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:49.699084 4902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:49.701264 4902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:49.701439 4902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:49.701730 4902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:49.702103 4902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:49.702165 4902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:49.702211 4902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:49.702256 4902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:49.708900 4902 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:44947
I20260809 03:32:49.708945 5080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:44947 every 8 connection(s)
I20260809 03:32:49.709343 4902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
Aug 09 03:32:49 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246369, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM
I20260809 03:32:49.718173 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 4902
I20260809 03:32:49.718319 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:32:49.718570 27810 external_mini_cluster.cc:1428] Setting key 501bcc2134b18ee57295f477ea6da7e1
I20260809 03:32:49.719385 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:49.710311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:36903 (local address 127.27.40.190:40413)
0809 03:32:49.710380 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:49.710382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:49.710573 (+ 191us) server_negotiation.cc:407] Connection header received
0809 03:32:49.711143 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:49.711145 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:49.711159 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:49.711211 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:49.712207 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:49.712792 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:49.713915 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:49.714123 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:49.714205 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:49.716405 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:49.716407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:49.716407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:49.716417 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:49.717847 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:49.718251 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:49.718252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:49.718252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:49.718334 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:49.718578 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:49.718578 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:49.718579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:49.718745 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:49.718823 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:49.719203 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 03:32:49.719296 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 03:32:49.719545 5083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:49.710293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.130:36903)
0809 03:32:49.710454 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:49.710489 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:49.711018 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:49.711289 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:49.711293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:49.711876 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:49.712073 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:49.712078 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:49.712889 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:49.712890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:49.713818 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:49.713820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:49.714223 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:49.714224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:49.714346 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:49.714870 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:49.714878 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:49.716306 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:49.717930 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:49.717936 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:49.717939 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:49.718163 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:49.718388 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:49.718389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:49.718389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:49.718499 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:49.718814 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:49.718821 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:49.719011 (+ 190us) client_negotiation.cc:769] Sending connection context
0809 03:32:49.719137 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 03:32:49.719307 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260809 03:32:49.720016 5081 heartbeater.cc:347] Connected to a master server at 127.27.40.190:40413
I20260809 03:32:49.720202 5081 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:49.720417 5081 heartbeater.cc:511] Master 127.27.40.190:40413 requested a full tablet report, sending...
I20260809 03:32:49.720831 4610 ts_manager.cc:194] Registered new tserver with Master: f20d19f3ea0c469f95ba10c2417139bb (127.27.40.130:44947)
I20260809 03:32:49.721422 4610 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:36903
WARNING: no policy specified for oryx/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.131.keytab.
Entry for principal oryx/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.131.keytab.
I20260809 03:32:49.746462 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40413
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.131.keytab
--principal=oryx/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:49.830910 5087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:49.831189 5087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:49.831228 5087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:49.832366 5087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:49.832440 5087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:49.832486 5087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:32:49.834048 5087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.131.keytab
--principal=oryx/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:49.834318 5087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:49.834555 5087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:49.839864 5094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:49.839921 5093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:49.840013 5100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:49.840135 5087 server_base.cc:1034] running on GCE node
I20260809 03:32:49.840342 5087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:49.840554 5087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:49.841696 5087 hybrid_clock.cc:648] HybridClock initialized: now 1786246369841672 us; error 34 us; skew 500 ppm
Aug 09 03:32:49 dist-test-slave-gkw7 krb5kdc[4540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246369, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:49.844601 5087 init.cc:377] Logged in from keytab as oryx/127.27.40.131@KRBTEST.COM (short username oryx)
I20260809 03:32:49.845093 5087 webserver.cc:460] Webserver started at http://127.27.40.131:43229/ using document root <none> and password file <none>
I20260809 03:32:49.845374 5087 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:49.845453 5087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:49.845645 5087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:49.846812 5087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3"
format_stamp: "Formatted at 2026-08-09 03:32:49 on dist-test-slave-gkw7"
server_key: "619493b72c0c78731041d3da49cb55b2"
server_key_iv: "6d6ca5e3af23014efcbc73d93bb50143"
server_key_version: "encryptionkey@0"
I20260809 03:32:49.847236 5087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3"
format_stamp: "Formatted at 2026-08-09 03:32:49 on dist-test-slave-gkw7"
server_key: "619493b72c0c78731041d3da49cb55b2"
server_key_iv: "6d6ca5e3af23014efcbc73d93bb50143"
server_key_version: "encryptionkey@0"
I20260809 03:32:49.848786 5087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:32:49.849713 5110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:49.849906 5087 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:49.850023 5087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3"
format_stamp: "Formatted at 2026-08-09 03:32:49 on dist-test-slave-gkw7"
server_key: "619493b72c0c78731041d3da49cb55b2"
server_key_iv: "6d6ca5e3af23014efcbc73d93bb50143"
server_key_version: "encryptionkey@0"
I20260809 03:32:49.850121 5087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:49.860757 5087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:49.863258 5087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:49.863431 5087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:49.863682 5087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:49.864002 5087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:49.864058 5087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:49.864102 5087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:49.864126 5087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:49.870565 5087 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:36575
I20260809 03:32:49.870615 5267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:36575 every 8 connection(s)
I20260809 03:32:49.870962 5087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:32:49.871824 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 5087
I20260809 03:32:49.871922 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:32:49.872149 27810 external_mini_cluster.cc:1428] Setting key 4bbeb99d062652593a6bf9f063e17f98
Aug 09 03:32:49 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246369, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM
I20260809 03:32:49.880476 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:49.871681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:55537 (local address 127.27.40.190:40413)
0809 03:32:49.871762 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:49.871765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:49.872038 (+ 273us) server_negotiation.cc:407] Connection header received
0809 03:32:49.872798 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:49.872799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:49.872813 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:49.872863 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:49.873978 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:49.874486 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:49.875443 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:49.875620 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:49.875677 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:49.877853 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:49.877854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:49.877855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:49.877861 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:49.879275 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:49.879636 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:49.879645 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:49.879645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:49.879696 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:49.879902 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:49.879903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:49.879903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:49.880003 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:49.880053 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:49.880339 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 03:32:49.880412 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:32:49.880553 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:49.871626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.131:55537)
0809 03:32:49.871855 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:49.871943 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:49.872696 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:49.872949 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:49.872953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:49.873701 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:49.873903 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:49.873908 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:49.874567 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:49.874568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:49.875372 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:49.875373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:49.875699 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:49.875699 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:49.875801 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:49.876294 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:49.876301 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:49.877772 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:49.879355 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:49.879362 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:49.879364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:49.879572 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:49.879753 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:49.879753 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:49.879754 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:49.879851 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:49.880057 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:49.880061 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:49.880195 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 03:32:49.880277 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 03:32:49.880395 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
I20260809 03:32:49.881088 5270 heartbeater.cc:347] Connected to a master server at 127.27.40.190:40413
I20260809 03:32:49.881191 5270 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:49.881371 5270 heartbeater.cc:511] Master 127.27.40.190:40413 requested a full tablet report, sending...
I20260809 03:32:49.881829 4610 ts_manager.cc:194] Registered new tserver with Master: b8c3f42a3e8a45ee9806e74efc6ef5f3 (127.27.40.131:36575)
I20260809 03:32:49.882201 4610 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:55537
I20260809 03:32:49.883769 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.27.40.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.40.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.40.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.189.keytab.
Entry for principal oryx/127.27.40.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.189.keytab.
I20260809 03:32:49.912982 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.189:44055
--webserver_interface=127.27.40.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.189.keytab
--principal=oryx/127.27.40.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:40413,127.27.40.189:44055
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:50.011128 5277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:50.011435 5277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:50.011463 5277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:50.012840 5277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:50.012892 5277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:50.012909 5277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:50.012923 5277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:50.012938 5277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:32:50.014657 5277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:40413,127.27.40.189:44055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.189.keytab
--principal=oryx/127.27.40.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.189:44055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.40.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
W20260809 03:32:50.014812 5277 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.27.40.190:40413,127.27.40.189:44055'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 03:32:50.015008 5277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:50.015283 5277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:50.020938 5293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:50.020984 5288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:50.021101 5290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:50.021389 5277 server_base.cc:1034] running on GCE node
I20260809 03:32:50.021708 5277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:50.022027 5277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:50.023206 5277 hybrid_clock.cc:648] HybridClock initialized: now 1786246370023185 us; error 36 us; skew 500 ppm
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:50.030316 5277 init.cc:377] Logged in from keytab as oryx/127.27.40.189@KRBTEST.COM (short username oryx)
I20260809 03:32:50.030810 5277 webserver.cc:460] Webserver started at http://127.27.40.189:44543/ using document root <none> and password file <none>
I20260809 03:32:50.031052 5277 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:50.031124 5277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:50.031284 5277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:50.032414 5277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/data/instance:
uuid: "35d92e21b47a498796999adcfc6ba6fb"
format_stamp: "Formatted at 2026-08-09 03:32:50 on dist-test-slave-gkw7"
server_key: "73a300df3bada7099c328ae52d5741b8"
server_key_iv: "9a3326636be38e7bf4f340151c6db365"
server_key_version: "encryptionkey@0"
I20260809 03:32:50.032804 5277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/wal/instance:
uuid: "35d92e21b47a498796999adcfc6ba6fb"
format_stamp: "Formatted at 2026-08-09 03:32:50 on dist-test-slave-gkw7"
server_key: "73a300df3bada7099c328ae52d5741b8"
server_key_iv: "9a3326636be38e7bf4f340151c6db365"
server_key_version: "encryptionkey@0"
I20260809 03:32:50.034400 5277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:32:50.035334 5303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:50.035573 5277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:50.035696 5277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/wal
uuid: "35d92e21b47a498796999adcfc6ba6fb"
format_stamp: "Formatted at 2026-08-09 03:32:50 on dist-test-slave-gkw7"
server_key: "73a300df3bada7099c328ae52d5741b8"
server_key_iv: "9a3326636be38e7bf4f340151c6db365"
server_key_version: "encryptionkey@0"
I20260809 03:32:50.035779 5277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:50.045331 5277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:50.047551 5277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:50.047741 5277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:50.054513 5277 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.189:44055
I20260809 03:32:50.054564 5375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.189:44055 every 8 connection(s)
I20260809 03:32:50.055071 5277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/data/info.pb
I20260809 03:32:50.056133 5376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:50.058308 5376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:32:50.059681 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 5277
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.189@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.40.189@KRBTEST.COM
I20260809 03:32:50.071230 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.059155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51026 (local address 127.27.40.190:40413)
0809 03:32:50.059259 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.059262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.059473 (+ 211us) server_negotiation.cc:407] Connection header received
0809 03:32:50.060344 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.060346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.060370 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.060475 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.061557 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.062641 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.064107 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.064462 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.064543 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.067440 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.067442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.067443 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.067452 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.069357 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.069940 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.069941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.069942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.070025 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.070340 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.070341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.070341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.070548 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.070634 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.070987 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.071102 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:32:50.071461 5378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.059115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51026)
0809 03:32:50.059340 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.059377 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.060127 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.060568 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.060572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.060994 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.061379 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.061386 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.062809 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.062810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.063978 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.063981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.064552 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.064553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.064702 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.065397 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.065408 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.067335 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.069484 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.069492 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.069495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.069833 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.070110 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.070111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.070112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.070266 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.070627 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.070633 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.070810 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.070937 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.071108 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":112,"threads_started":1}
I20260809 03:32:50.072117 5382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.060241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54044 (local address 127.27.40.189:44055)
0809 03:32:50.060571 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.060575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.060590 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:32:50.060612 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.060614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.060633 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.060700 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.061572 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.063145 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.064169 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.064714 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.064772 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.068099 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.068107 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.068107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.068116 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.070170 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.070577 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.070581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.070581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.070653 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.070880 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.070881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.070881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.071101 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.071320 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.071834 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.071948 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":222,"threads_started":1}
I20260809 03:32:50.072279 5376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.189@KRBTEST.COM for oryx/127.27.40.189@KRBTEST.COM
I20260809 03:32:50.079347 5378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.072704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:44055 (local address 127.0.0.1:54054)
0809 03:32:50.072741 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.072753 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.072796 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.072961 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.072961 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.073091 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.073127 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.073128 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.073954 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.073954 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.074501 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.074503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.074930 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.074931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.074981 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.075358 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.075362 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.076658 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.078427 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.078428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.078428 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.078619 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.078796 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.078797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.078797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.078850 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.079073 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.079074 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.079124 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.079186 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.079264 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 03:32:50.079398 5382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.072739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54054 (local address 127.27.40.189:44055)
0809 03:32:50.072774 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.072776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.072785 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:50.072881 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.072882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.072895 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.072957 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.073192 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.073843 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.074588 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.074852 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.074914 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.076743 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.076745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.076745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.076751 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.078348 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.078686 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.078687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.078687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.078737 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.078891 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.078891 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.078891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.078979 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.079027 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.079217 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.079310 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":15}
I20260809 03:32:50.080974 5376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: Bootstrap starting.
I20260809 03:32:50.082356 5376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:50.082913 5376 log.cc:824] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:50.083808 5376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: No bootstrap required, opened a new log
I20260809 03:32:50.086282 5376 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } }
I20260809 03:32:50.086406 5376 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:50.086429 5376 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35d92e21b47a498796999adcfc6ba6fb, State: Initialized, Role: FOLLOWER
I20260809 03:32:50.086519 5376 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } }
I20260809 03:32:50.086860 5386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } } }
I20260809 03:32:50.086982 5376 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:50.087044 5386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: This master's current role is: FOLLOWER
W20260809 03:32:50.089524 5403 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:32:50.089617 5403 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:32:50.091114 5277 master_runner.cc:428] Detected that this master 35d92e21b47a498796999adcfc6ba6fb is joining an existing cluster
I20260809 03:32:50.091179 5277 master_runner.cc:432] Initiating AddMaster RPC to add 127.27.40.189:44055
I20260809 03:32:50.102415 5382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.093368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54064 (local address 127.27.40.189:44055)
0809 03:32:50.093426 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.093429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.094100 (+ 671us) server_negotiation.cc:407] Connection header received
0809 03:32:50.094134 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.094135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.094148 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.094238 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.094750 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.095248 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.096703 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.096931 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.096963 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.098892 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.098893 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.098893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.098900 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.100713 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.101217 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.101218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.101219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.101275 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.101490 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.101491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.101491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.101691 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.101755 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.102113 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.102278 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":23}
I20260809 03:32:50.102442 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.093153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:44055 (local address 127.0.0.1:54064)
0809 03:32:50.093942 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.093954 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.094052 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.094248 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.094249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.094554 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.094664 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.094666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.095330 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.095331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.096605 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.096608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.097019 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.097020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.097082 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.097970 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.097976 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.098778 (+ 802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.100808 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.100809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.100809 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.101109 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.101348 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.101349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.101349 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.101407 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.101776 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.101777 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.101889 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.102059 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.102205 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":127,"threads_started":1}
I20260809 03:32:50.103037 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.092835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51028 (local address 127.27.40.190:40413)
0809 03:32:50.092911 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.092913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.093103 (+ 190us) server_negotiation.cc:407] Connection header received
0809 03:32:50.093183 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.093184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.093197 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.093306 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.094315 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.095348 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.096729 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.097073 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.097158 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.098845 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.098847 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.098847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.098854 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.101128 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.101617 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.101619 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.101619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.101674 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.102060 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.102062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.102062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.102220 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.102287 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.102818 (+ 531us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.102901 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:32:50.103355 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.092743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51028)
0809 03:32:50.093030 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.093042 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.093100 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.093312 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.093314 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.093678 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.094189 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.094191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.095438 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.095440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.096620 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.096622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.097182 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.097183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.097303 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.097970 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.097976 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.098738 (+ 762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.101241 (+ 2503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.101242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.101243 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.101486 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.101771 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.101772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.101773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.101945 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.102319 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.102320 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.102363 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.102447 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.103213 (+ 766us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":177,"threads_started":1}
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246369, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.190@KRBTEST.COM for oryx/127.27.40.189@KRBTEST.COM
I20260809 03:32:50.115100 5382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.106860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54074 (local address 127.27.40.189:44055)
0809 03:32:50.106909 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.106911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.107115 (+ 204us) server_negotiation.cc:407] Connection header received
0809 03:32:50.107218 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.107218 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.107227 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.107275 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.107953 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.108483 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.109768 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.109947 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.110000 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.112331 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.112333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.112333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.112341 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.113835 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.114237 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.114238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.114239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.114297 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.114522 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.114523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.114524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.114636 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.114717 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.114900 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.114985 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 03:32:50.115157 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.106761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:44055 (local address 127.0.0.1:54074)
0809 03:32:50.106980 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.107038 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.107131 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.107282 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.107286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.107793 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.107866 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.107869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.108578 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.108580 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.109686 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.109689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.110029 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.110030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.110126 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.110627 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.110632 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.112235 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.113920 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.113928 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.113931 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.114139 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.114368 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.114369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.114370 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.114444 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.114710 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.114717 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.114771 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.114874 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.114992 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":102,"threads_started":1}
I20260809 03:32:50.115994 4610 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.27.40.189:44055: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb"
member_type: NON_VOTER
last_known_addr {
host: "127.27.40.189"
port: 44055
}
attrs {
promote: true
}
}
dest_uuid: "0b478a01c8d24855948ebf23178e3c8f"
cas_config_opid_index: -1
I20260809 03:32:50.116365 4610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } }
I20260809 03:32:50.116880 5419 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 35d92e21b47a498796999adcfc6ba6fb (127.27.40.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } } }
I20260809 03:32:50.117345 5340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:50.118194 5419 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.27.40.189:44055
I20260809 03:32:50.118281 5419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } } }
I20260809 03:32:50.118355 5419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:50.118458 5277 master.cc:537] Master@127.27.40.189:44055 shutting down...
I20260809 03:32:50.118727 5340 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 FOLLOWER]: Refusing update from remote peer 0b478a01c8d24855948ebf23178e3c8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 03:32:50.119083 5419 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 03:32:50.120090 4588 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 03:32:50.120512 4588 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f -> Peer 35d92e21b47a498796999adcfc6ba6fb (127.27.40.189:44055): Couldn't send request to peer 35d92e21b47a498796999adcfc6ba6fb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 03:32:50.121097 5277 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:50.121186 5277 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:50.121230 5277 tablet_replica.cc:330] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: stopping tablet replica
I20260809 03:32:50.134428 5277 master.cc:552] Master@127.27.40.189:44055 shutdown complete.
I20260809 03:32:50.134503 5277 master_runner.cc:305] Clearing existing system tablet
I20260809 03:32:50.134642 5277 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:50.134714 5277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:50.135366 5277 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260809 03:32:50.135939 5423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:50.136093 5277 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:50.136150 5277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:50.136174 5277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/wal
uuid: "35d92e21b47a498796999adcfc6ba6fb"
format_stamp: "Formatted at 2026-08-09 03:32:50 on dist-test-slave-gkw7"
server_key: "73a300df3bada7099c328ae52d5741b8"
server_key_iv: "9a3326636be38e7bf4f340151c6db365"
server_key_version: "encryptionkey@0"
I20260809 03:32:50.136818 5277 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:50.138192 5277 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 03:32:50.138252 5277 log.cc:1196] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 03:32:50.138469 5277 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: Deleting consensus metadata
I20260809 03:32:50.138609 5277 master_runner.cc:315] Copying system tablet from 127.27.40.190:40413
I20260809 03:32:50.139353 5277 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: tablet copy: Beginning tablet copy session from remote peer at address 127.27.40.190:40413
I20260809 03:32:50.145778 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.139779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51044 (local address 127.27.40.190:40413)
0809 03:32:50.139834 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.139837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.139892 (+ 55us) server_negotiation.cc:407] Connection header received
0809 03:32:50.139977 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.139978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.139988 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.140048 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.140442 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.140953 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.141891 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.142103 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.142197 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.143177 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.143178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.143179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.143186 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.144559 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.144932 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.144933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.144934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.144997 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.145203 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.145204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.145205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.145325 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.145405 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.145598 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.145681 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 03:32:50.145797 5434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.139696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51044)
0809 03:32:50.139817 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.139829 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.139911 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.140054 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.140055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.140250 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.140350 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.140352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.141039 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.141041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.141799 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.141801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.142199 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.142200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.142275 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.142778 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.142783 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.143076 (+ 293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.144666 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.144667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.144667 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.144845 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.145066 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.145067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.145068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.145134 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.145396 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.145396 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.145462 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.145569 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.145688 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":70,"threads_started":1}
I20260809 03:32:50.146111 4643 tablet_copy_service.cc:140] P 0b478a01c8d24855948ebf23178e3c8f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 35d92e21b47a498796999adcfc6ba6fb ({username='oryx', principal='oryx/127.27.40.189@KRBTEST.COM'} at 127.0.0.1:51044)
I20260809 03:32:50.146216 4643 tablet_copy_service.cc:161] P 0b478a01c8d24855948ebf23178e3c8f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 35d92e21b47a498796999adcfc6ba6fb at {username='oryx', principal='oryx/127.27.40.189@KRBTEST.COM'} at 127.0.0.1:51044: session id = 35d92e21b47a498796999adcfc6ba6fb-00000000000000000000000000000000
I20260809 03:32:50.146850 4643 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 03:32:50.147472 5277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 03:32:50.148396 5277 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: tablet copy: Starting download of 0 data blocks...
I20260809 03:32:50.148551 5277 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: tablet copy: Starting download of 1 WAL segments...
I20260809 03:32:50.149406 5277 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 03:32:50.150324 4643 tablet_copy_service.cc:342] P 0b478a01c8d24855948ebf23178e3c8f: Request end of tablet copy session 35d92e21b47a498796999adcfc6ba6fb-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.27.40.189@KRBTEST.COM'} at 127.0.0.1:51044
I20260809 03:32:50.150384 4643 tablet_copy_service.cc:433] P 0b478a01c8d24855948ebf23178e3c8f: ending tablet copy session 35d92e21b47a498796999adcfc6ba6fb-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 35d92e21b47a498796999adcfc6ba6fb
I20260809 03:32:50.152917 5277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:50.154145 5437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:50.154168 5439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:50.154398 5441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:50.154814 5277 server_base.cc:1034] running on GCE node
I20260809 03:32:50.154908 5277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:50.155041 5277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:50.156186 5277 hybrid_clock.cc:648] HybridClock initialized: now 1786246370156175 us; error 22 us; skew 500 ppm
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:50.157673 5277 init.cc:377] Logged in from keytab as oryx/127.27.40.189@KRBTEST.COM (short username oryx)
I20260809 03:32:50.157951 5277 webserver.cc:460] Webserver started at http://127.27.40.189:45905/ using document root <none> and password file <none>
I20260809 03:32:50.158089 5277 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:50.158121 5277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:50.159556 5277 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:32:50.160367 5452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:50.160511 5277 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:32:50.160591 5277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/wal
uuid: "35d92e21b47a498796999adcfc6ba6fb"
format_stamp: "Formatted at 2026-08-09 03:32:50 on dist-test-slave-gkw7"
server_key: "73a300df3bada7099c328ae52d5741b8"
server_key_iv: "9a3326636be38e7bf4f340151c6db365"
server_key_version: "encryptionkey@0"
I20260809 03:32:50.160707 5277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:50.169086 5277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:50.171250 5277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:50.175019 5277 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.189:44055
I20260809 03:32:50.175439 5533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.189:44055 every 8 connection(s)
I20260809 03:32:50.175468 5277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-1/data/info.pb
I20260809 03:32:50.176355 5534 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:32:50.176764 5534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: Bootstrap starting.
I20260809 03:32:50.179872 5534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:32:50.180172 5534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: Bootstrap complete.
I20260809 03:32:50.180419 5534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } }
I20260809 03:32:50.180529 5534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 35d92e21b47a498796999adcfc6ba6fb, State: Initialized, Role: LEARNER
I20260809 03:32:50.180578 5534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } }
I20260809 03:32:50.180745 5534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:50.180750 5539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } } }
I20260809 03:32:50.181020 5539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: This master's current role is: LEARNER
I20260809 03:32:50.182511 5555 catalog_manager.cc:1259] Loaded cluster ID: e78f5199826845eeb37908ef76813cbe
I20260809 03:32:50.182564 5555 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: loading cluster ID for follower catalog manager: success
I20260809 03:32:50.183668 5555 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: acquiring CA information for follower catalog manager: success
I20260809 03:32:50.183974 5555 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:32:50.233857 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.224426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54096 (local address 127.27.40.189:44055)
0809 03:32:50.224649 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.224652 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.224664 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:32:50.224700 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.224701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.224717 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.224821 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.225603 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.226807 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.227955 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.228286 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.228378 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.230110 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.230111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.230112 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.230119 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.232435 (+ 2316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.232944 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.232945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.232945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.233025 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.233264 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.233264 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.233265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.233381 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.233464 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.233640 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.233726 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":130,"threads_started":1}
WARNING: no policy specified for oryx/127.27.40.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.40.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.40.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.188.keytab.
Entry for principal oryx/127.27.40.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.188.keytab.
I20260809 03:32:50.269158 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.188:44133
--webserver_interface=127.27.40.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.188.keytab
--principal=oryx/127.27.40.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:40413,127.27.40.189:44055,127.27.40.188:44133
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:50.373080 5562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:50.373292 5562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:50.373314 5562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:50.374517 5562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:50.374564 5562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:50.374578 5562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:50.374591 5562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:50.374603 5562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:32:50.376284 5562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:40413,127.27.40.189:44055,127.27.40.188:44133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.188.keytab
--principal=oryx/127.27.40.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.188:44133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.40.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:50.376494 5562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:50.376667 5562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:50.382668 5569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:50.382690 5573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:50.382696 5570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:50.383056 5562 server_base.cc:1034] running on GCE node
I20260809 03:32:50.383272 5562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:50.383553 5562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:50.384728 5562 hybrid_clock.cc:648] HybridClock initialized: now 1786246370384711 us; error 32 us; skew 500 ppm
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:50.388612 5562 init.cc:377] Logged in from keytab as oryx/127.27.40.188@KRBTEST.COM (short username oryx)
I20260809 03:32:50.389102 5562 webserver.cc:460] Webserver started at http://127.27.40.188:42391/ using document root <none> and password file <none>
I20260809 03:32:50.389365 5562 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:50.389437 5562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:50.389626 5562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:50.390832 5562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/data/instance:
uuid: "b505344d51174a9c9651e17bb1061ae0"
format_stamp: "Formatted at 2026-08-09 03:32:50 on dist-test-slave-gkw7"
server_key: "62057ce8d24fcf20dca2233993e6883c"
server_key_iv: "14d7b1ac7f58ae7d3b4780d67c8d84cd"
server_key_version: "encryptionkey@0"
I20260809 03:32:50.391237 5562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/wal/instance:
uuid: "b505344d51174a9c9651e17bb1061ae0"
format_stamp: "Formatted at 2026-08-09 03:32:50 on dist-test-slave-gkw7"
server_key: "62057ce8d24fcf20dca2233993e6883c"
server_key_iv: "14d7b1ac7f58ae7d3b4780d67c8d84cd"
server_key_version: "encryptionkey@0"
I20260809 03:32:50.392761 5562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:32:50.393733 5583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:50.393931 5562 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:50.394035 5562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/wal
uuid: "b505344d51174a9c9651e17bb1061ae0"
format_stamp: "Formatted at 2026-08-09 03:32:50 on dist-test-slave-gkw7"
server_key: "62057ce8d24fcf20dca2233993e6883c"
server_key_iv: "14d7b1ac7f58ae7d3b4780d67c8d84cd"
server_key_version: "encryptionkey@0"
I20260809 03:32:50.394145 5562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:50.403577 5562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:50.406297 5562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:50.406431 5562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:50.411806 5562 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.188:44133
I20260809 03:32:50.411859 5654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.188:44133 every 8 connection(s)
I20260809 03:32:50.412242 5562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/data/info.pb
I20260809 03:32:50.413302 5655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:50.415006 5655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:32:50.416251 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 5562
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.188@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.40.188@KRBTEST.COM
I20260809 03:32:50.426607 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.415867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51058 (local address 127.27.40.190:40413)
0809 03:32:50.416108 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.416111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.416305 (+ 194us) server_negotiation.cc:407] Connection header received
0809 03:32:50.417401 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.417403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.417418 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.417486 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.418134 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.418642 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.419834 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.420184 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.420251 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.422976 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.422977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.422978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.422985 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.424962 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.425448 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.425449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.425450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.425521 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.425883 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.425884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.425885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.426028 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.426118 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.426412 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.426504 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260809 03:32:50.426864 5658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.415719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51058)
0809 03:32:50.416165 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.416203 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.417275 (+ 1072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.417516 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.417518 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.417814 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.418037 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.418042 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.418829 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.418830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.419749 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.419751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.420279 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.420280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.420369 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.421001 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.421012 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.422883 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.425077 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.425083 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.425084 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.425328 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.425651 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.425652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.425652 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.425764 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.426122 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.426130 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.426266 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.426384 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.426540 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":194,"threads_started":1}
I20260809 03:32:50.427321 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.416859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36126 (local address 127.27.40.188:44133)
0809 03:32:50.417164 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.417169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.417224 (+ 55us) server_negotiation.cc:407] Connection header received
0809 03:32:50.417306 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.417313 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.417337 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.417425 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.418186 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.419218 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.420003 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.420441 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.420527 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.423685 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.423695 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.423695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.423702 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.425709 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.426071 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.426075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.426075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.426136 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.426365 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.426365 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.426366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.426510 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.427043 (+ 533us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.427090 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.427168 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":174,"threads_started":1}
I20260809 03:32:50.427984 5655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.188@KRBTEST.COM for oryx/127.27.40.189@KRBTEST.COM
I20260809 03:32:50.434641 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.428339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54106 (local address 127.27.40.189:44055)
0809 03:32:50.428383 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.428386 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.428393 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:32:50.428418 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.428419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.428439 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.428501 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.428782 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.429293 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.430062 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.430285 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.430365 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.432112 (+ 1747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.432114 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.432114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.432122 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.433399 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.433812 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.433813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.433813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.433876 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.434109 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.434110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.434111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.434228 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.434305 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.434466 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.434551 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 03:32:50.434643 5658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.428237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:44055 (local address 127.0.0.1:54106)
0809 03:32:50.428289 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.428300 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.428344 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.428505 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.428506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.428657 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.428690 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.428691 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.429379 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.429380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.429975 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.429977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.430361 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.430362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.430422 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.430800 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.430805 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.431990 (+ 1185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.433519 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.433520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.433520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.433734 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.433946 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.433947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.433948 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.434014 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.434308 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.434309 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.434351 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.434439 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.434537 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 03:32:50.435228 5655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.188@KRBTEST.COM for oryx/127.27.40.188@KRBTEST.COM
I20260809 03:32:50.441720 5658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.435564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:44133 (local address 127.0.0.1:36134)
0809 03:32:50.435632 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.435645 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.435695 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.435797 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.435798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.435905 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.435935 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.435936 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.436613 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.436614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.437137 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.437139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.437509 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.437510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.437619 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.438005 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.438010 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.439203 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.440846 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.440847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.440847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.441005 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.441188 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.441188 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.441189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.441239 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.441437 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.441438 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.441471 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.441520 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.441658 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 03:32:50.441790 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.435608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36134 (local address 127.27.40.188:44133)
0809 03:32:50.435642 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.435644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.435679 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:32:50.435730 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.435731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.435739 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.435783 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.435984 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.436498 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.437218 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.437444 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.437496 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.439292 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.439294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.439294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.439300 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.440768 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.441076 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.441077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.441077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.441127 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.441291 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.441291 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.441292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.441385 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.441433 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.441597 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.441703 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":17}
I20260809 03:32:50.443387 5655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: Bootstrap starting.
I20260809 03:32:50.444380 5655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:50.444906 5655 log.cc:824] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:50.445781 5655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: No bootstrap required, opened a new log
I20260809 03:32:50.447371 5655 raft_consensus.cc:348] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } }
I20260809 03:32:50.447528 5655 raft_consensus.cc:374] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:50.447595 5655 raft_consensus.cc:729] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b505344d51174a9c9651e17bb1061ae0, State: Initialized, Role: FOLLOWER
I20260809 03:32:50.447739 5655 consensus_queue.cc:260] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } }
I20260809 03:32:50.448052 5655 sys_catalog.cc:564] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:50.448083 5662 sys_catalog.cc:455] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } } }
I20260809 03:32:50.448203 5662 sys_catalog.cc:458] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 03:32:50.449775 5680 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:32:50.449857 5680 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:32:50.452227 5562 master_runner.cc:428] Detected that this master b505344d51174a9c9651e17bb1061ae0 is joining an existing cluster
I20260809 03:32:50.452275 5562 master_runner.cc:432] Initiating AddMaster RPC to add 127.27.40.188:44133
I20260809 03:32:50.474491 5692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.453565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51074)
0809 03:32:50.453876 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.453888 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.453973 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.454120 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.454121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.454481 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.454637 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.454639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.455335 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.455336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.455940 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.455941 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.456352 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.456353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.456417 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.456885 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.456890 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.457238 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.459079 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.459080 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.459080 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.459320 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.459740 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.459741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.459741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.473676 (+ 13935us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.474055 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.474056 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.474103 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.474232 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.474364 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":92,"threads_started":1}
I20260809 03:32:50.474503 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.453602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51074 (local address 127.27.40.190:40413)
0809 03:32:50.453657 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.453659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.453943 (+ 284us) server_negotiation.cc:407] Connection header received
0809 03:32:50.454033 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.454034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.454047 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.454115 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.454727 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.455253 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.456039 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.456255 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.456329 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.457313 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.457314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.457314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.457323 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.458984 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.459611 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.459612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.459613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.459663 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.473763 (+ 14100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.473765 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.473765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.473962 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.474046 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.474290 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.474388 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 03:32:50.476100 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.453622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36138 (local address 127.27.40.188:44133)
0809 03:32:50.453663 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.453665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.454269 (+ 604us) server_negotiation.cc:407] Connection header received
0809 03:32:50.454351 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.454352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.454361 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.454418 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.454814 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.455555 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.456649 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.456927 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.456965 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.459407 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.459409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.459409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.459416 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.460859 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.474856 (+ 13997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.474856 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.474857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.474911 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.475533 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.475534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.475534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.475697 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.475776 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.475934 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.476013 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 03:32:50.476666 5693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.453641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:44133 (local address 127.0.0.1:36138)
0809 03:32:50.454216 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.454226 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.454289 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.454424 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.454424 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.454635 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.454711 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.454713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.455659 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.455659 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.456524 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.456527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.457030 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.457031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.457084 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.457669 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.457695 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.459304 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.460940 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.460941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.460941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.474785 (+ 13844us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.475351 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.475352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.475352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.475416 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.475800 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.475801 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.475840 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.476401 (+ 561us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.476551 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":98,"threads_started":1}
I20260809 03:32:50.477856 5691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.453921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:44055 (local address 127.0.0.1:54112)
0809 03:32:50.461672 (+ 7751us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.461681 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.461731 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.461881 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.461882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.462022 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.462372 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.462374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.463395 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.463396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.463940 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.463942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.464398 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.464399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.464462 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.464858 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.464863 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.474378 (+ 9515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.476366 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.476367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.476367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.476659 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.476914 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.476914 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.476915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.476999 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.477467 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.477468 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.477523 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.477637 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.477745 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7727,"thread_start_us":82,"threads_started":1}
I20260809 03:32:50.477871 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.453864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54112 (local address 127.27.40.189:44055)
0809 03:32:50.453930 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.453933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.461742 (+ 7809us) server_negotiation.cc:407] Connection header received
0809 03:32:50.461773 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.461774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.461799 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.461865 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.462458 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.463299 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.464023 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.464321 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.464393 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.474479 (+ 10086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.474480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.474481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.474488 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.476235 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.476770 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.476772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.476772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.476835 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.477160 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.477161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.477162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.477318 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.477367 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.477686 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.477777 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 03:32:50.478731 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 4709
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246369, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.190@KRBTEST.COM for oryx/127.27.40.188@KRBTEST.COM
I20260809 03:32:50.484632 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.477386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:44133 (local address 127.0.0.1:36148)
0809 03:32:50.477432 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.477443 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.477478 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.477696 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.477697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.477858 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.477910 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.477912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.478843 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.478844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.479585 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.479586 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.480007 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.480008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.480093 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.480494 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.480499 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.481965 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.483618 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.483619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.483619 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.483840 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.484011 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.484012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.484012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.484062 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.484314 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.484315 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.484354 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.484406 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.484505 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 03:32:50.484917 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.477455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36148 (local address 127.27.40.188:44133)
0809 03:32:50.477524 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.477526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.477532 (+ 6us) server_negotiation.cc:407] Connection header received
0809 03:32:50.477582 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.477583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.477593 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.477644 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.478000 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.478726 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.479663 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.479918 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.479967 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.482047 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.482048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.482049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.482055 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.483537 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.483909 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.483910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.483911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.483957 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.484126 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.484126 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.484127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.484244 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.484293 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.484735 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.484805 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:32:50.488005 4610 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.27.40.188:44133: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b505344d51174a9c9651e17bb1061ae0"
member_type: NON_VOTER
last_known_addr {
host: "127.27.40.188"
port: 44133
}
attrs {
promote: true
}
}
dest_uuid: "0b478a01c8d24855948ebf23178e3c8f"
cas_config_opid_index: 5
I20260809 03:32:50.488292 4610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } }
I20260809 03:32:50.488893 5417 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b505344d51174a9c9651e17bb1061ae0 (127.27.40.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } } }
I20260809 03:32:50.490201 5417 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.27.40.188:44133
I20260809 03:32:50.490278 5417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } } }
I20260809 03:32:50.490370 5417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:50.490856 5619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:50.492180 5619 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 FOLLOWER]: Refusing update from remote peer 0b478a01c8d24855948ebf23178e3c8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 03:32:50.492463 5419 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:50.493206 5562 master.cc:537] Master@127.27.40.188:44133 shutting down...
I20260809 03:32:50.493986 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:39613
--local_ip_for_outbound_sockets=127.27.40.129
--tserver_master_addrs=127.27.40.190:40413
--webserver_port=36057
--webserver_interface=127.27.40.129
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.129.keytab
--principal=oryx/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.40.190:40413,127.27.40.189:44055
--tserver_master_addrs=127.27.40.190:40413,127.27.40.189:44055,127.27.40.188:44133 with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:32:50.496500 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.488820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:44055 (local address 127.0.0.1:54120)
0809 03:32:50.490605 (+ 1785us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.490617 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.490655 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.492642 (+ 1987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.492643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.492645 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:50.492691 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.492693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.493882 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.493883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.495236 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.495240 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.496194 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.496195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.496268 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.496311 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.496363 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.496409 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1769}
I20260809 03:32:50.496555 5619 raft_consensus.cc:858] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } }
W20260809 03:32:50.496640 5619 raft_consensus.cc:1565] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } }
I20260809 03:32:50.496797 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.488896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54120 (local address 127.27.40.189:44055)
0809 03:32:50.488936 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.488937 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.492357 (+ 3420us) server_negotiation.cc:407] Connection header received
0809 03:32:50.492475 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.492476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.492504 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.492559 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:50.492833 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.493787 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.495337 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.496013 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.496082 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.496157 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.496660 (+ 503us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.496707 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 03:32:50.497015 5490 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 LEARNER]: Refusing update from remote peer 0b478a01c8d24855948ebf23178e3c8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 03:32:50.497215 5419 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 03:32:50.497768 5490 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b505344d51174a9c9651e17bb1061ae0 (127.27.40.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } } }
I20260809 03:32:50.497963 5619 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 35d92e21b47a498796999adcfc6ba6fb (127.27.40.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } } }
I20260809 03:32:50.498385 5695 mvcc.cc:205] Tried to move back new op lower bound from 7316465128996519936 to 7316465128859418624. Current Snapshot: MvccSnapshot[applied={T|T < 7316465128868274176}]
I20260809 03:32:50.498989 5539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b478a01c8d24855948ebf23178e3c8f. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } } }
I20260809 03:32:50.499090 5539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: This master's current role is: LEARNER
I20260809 03:32:50.499151 5539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } } }
I20260809 03:32:50.499209 5539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: This master's current role is: LEARNER
I20260809 03:32:50.499202 5662 sys_catalog.cc:455] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b478a01c8d24855948ebf23178e3c8f. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } } }
I20260809 03:32:50.499274 5662 sys_catalog.cc:458] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 03:32:50.499310 5662 sys_catalog.cc:455] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } } }
I20260809 03:32:50.499347 5662 sys_catalog.cc:458] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 03:32:50.499374 5662 sys_catalog.cc:455] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: NON_VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: true } } }
I20260809 03:32:50.499408 5662 sys_catalog.cc:458] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 03:32:50.499692 5419 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f: attempting to promote NON_VOTER 35d92e21b47a498796999adcfc6ba6fb to VOTER
I20260809 03:32:50.499897 5419 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } }
I20260809 03:32:50.500509 5490 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 LEARNER]: Refusing update from remote peer 0b478a01c8d24855948ebf23178e3c8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 03:32:50.500710 5417 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260809 03:32:50.500802 4585 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f -> Peer b505344d51174a9c9651e17bb1061ae0 (127.27.40.188:44133): Couldn't send request to peer b505344d51174a9c9651e17bb1061ae0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 03:32:50.501349 5417 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 35d92e21b47a498796999adcfc6ba6fb (127.27.40.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } } }
I20260809 03:32:50.501628 5490 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 35d92e21b47a498796999adcfc6ba6fb (127.27.40.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } } }
I20260809 03:32:50.502405 5417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } } }
I20260809 03:32:50.502489 5417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:50.502547 5417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } } }
I20260809 03:32:50.502617 5417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:50.502815 5708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } } }
I20260809 03:32:50.502969 5708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:32:50.510968 5562 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260809 03:32:50.511169 5562 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260809 03:32:50.511242 5562 tablet_replica.cc:330] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: stopping tablet replica
I20260809 03:32:50.524901 5562 master.cc:552] Master@127.27.40.188:44133 shutdown complete.
I20260809 03:32:50.524964 5562 master_runner.cc:305] Clearing existing system tablet
I20260809 03:32:50.525111 5562 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:50.525164 5562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:50.525938 5562 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 03:32:50.526535 5712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:50.526695 5562 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:50.526757 5562 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:32:50.526778 5562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/wal
uuid: "b505344d51174a9c9651e17bb1061ae0"
format_stamp: "Formatted at 2026-08-09 03:32:50 on dist-test-slave-gkw7"
server_key: "62057ce8d24fcf20dca2233993e6883c"
server_key_iv: "14d7b1ac7f58ae7d3b4780d67c8d84cd"
server_key_version: "encryptionkey@0"
I20260809 03:32:50.527463 5562 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:50.528995 5562 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 03:32:50.529079 5562 log.cc:1196] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 03:32:50.529318 5562 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: Deleting consensus metadata
I20260809 03:32:50.529495 5562 master_runner.cc:315] Copying system tablet from 127.27.40.190:40413
I20260809 03:32:50.530205 5562 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: tablet copy: Beginning tablet copy session from remote peer at address 127.27.40.190:40413
I20260809 03:32:50.537694 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.530611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51076 (local address 127.27.40.190:40413)
0809 03:32:50.530661 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.530664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.530716 (+ 52us) server_negotiation.cc:407] Connection header received
0809 03:32:50.530788 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.530789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.530801 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.530888 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.531279 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.532061 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.533091 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.533384 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.533467 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.534660 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.534662 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.534662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.534671 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.536401 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.536803 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.536805 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.536805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.536871 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.537095 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.537096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.537096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.537234 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.537304 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.537486 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.537597 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 03:32:50.537712 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.530527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51076)
0809 03:32:50.530657 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.530668 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.530730 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.530902 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.530903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.531124 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.531178 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.531180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.532160 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.532161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.532990 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.532993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.533474 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.533474 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.533558 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.534120 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.534125 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.534553 (+ 428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.536493 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.536494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.536495 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.536724 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.536941 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.536942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.536942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.537015 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.537304 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.537305 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.537357 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.537440 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.537617 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260809 03:32:50.538339 4643 tablet_copy_service.cc:140] P 0b478a01c8d24855948ebf23178e3c8f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b505344d51174a9c9651e17bb1061ae0 ({username='oryx', principal='oryx/127.27.40.188@KRBTEST.COM'} at 127.0.0.1:51076)
I20260809 03:32:50.538409 4643 tablet_copy_service.cc:161] P 0b478a01c8d24855948ebf23178e3c8f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b505344d51174a9c9651e17bb1061ae0 at {username='oryx', principal='oryx/127.27.40.188@KRBTEST.COM'} at 127.0.0.1:51076: session id = b505344d51174a9c9651e17bb1061ae0-00000000000000000000000000000000
I20260809 03:32:50.539009 4643 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 03:32:50.539553 5562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 03:32:50.540958 5562 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: tablet copy: Starting download of 0 data blocks...
I20260809 03:32:50.541126 5562 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: tablet copy: Starting download of 1 WAL segments...
I20260809 03:32:50.541970 5562 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 03:32:50.542862 4643 tablet_copy_service.cc:342] P 0b478a01c8d24855948ebf23178e3c8f: Request end of tablet copy session b505344d51174a9c9651e17bb1061ae0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.27.40.188@KRBTEST.COM'} at 127.0.0.1:51076
I20260809 03:32:50.542918 4643 tablet_copy_service.cc:433] P 0b478a01c8d24855948ebf23178e3c8f: ending tablet copy session b505344d51174a9c9651e17bb1061ae0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b505344d51174a9c9651e17bb1061ae0
I20260809 03:32:50.545444 5562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:50.546695 5720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:50.546741 5723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:50.547029 5562 server_base.cc:1034] running on GCE node
W20260809 03:32:50.547144 5721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:50.547297 5562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:50.547427 5562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:50.548552 5562 hybrid_clock.cc:648] HybridClock initialized: now 1786246370548527 us; error 43 us; skew 500 ppm
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:50.550129 5562 init.cc:377] Logged in from keytab as oryx/127.27.40.188@KRBTEST.COM (short username oryx)
I20260809 03:32:50.550392 5562 webserver.cc:460] Webserver started at http://127.27.40.188:35597/ using document root <none> and password file <none>
I20260809 03:32:50.550527 5562 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:50.550590 5562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:50.551323 5562 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:50.551996 5730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:50.552156 5562 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 03:32:50.552208 5562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/wal
uuid: "b505344d51174a9c9651e17bb1061ae0"
format_stamp: "Formatted at 2026-08-09 03:32:50 on dist-test-slave-gkw7"
server_key: "62057ce8d24fcf20dca2233993e6883c"
server_key_iv: "14d7b1ac7f58ae7d3b4780d67c8d84cd"
server_key_version: "encryptionkey@0"
I20260809 03:32:50.552258 5562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:50.568958 5562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:50.571146 5562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:50.575001 5562 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.188:44133
I20260809 03:32:50.575248 5801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.188:44133 every 8 connection(s)
I20260809 03:32:50.575410 5562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/master-2/data/info.pb
I20260809 03:32:50.576375 5803 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:32:50.576927 5803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: Bootstrap starting.
I20260809 03:32:50.579594 5803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:32:50.579862 5803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: Bootstrap complete.
I20260809 03:32:50.580058 5803 raft_consensus.cc:348] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } }
I20260809 03:32:50.580152 5803 raft_consensus.cc:729] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b505344d51174a9c9651e17bb1061ae0, State: Initialized, Role: LEARNER
I20260809 03:32:50.580190 5803 consensus_queue.cc:260] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } }
I20260809 03:32:50.580437 5803 sys_catalog.cc:564] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:50.581614 5806 sys_catalog.cc:455] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } } }
I20260809 03:32:50.581861 5806 sys_catalog.cc:458] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: This master's current role is: LEARNER
I20260809 03:32:50.582623 5820 catalog_manager.cc:1259] Loaded cluster ID: e78f5199826845eeb37908ef76813cbe
I20260809 03:32:50.582669 5820 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: loading cluster ID for follower catalog manager: success
I20260809 03:32:50.583698 5820 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: acquiring CA information for follower catalog manager: success
I20260809 03:32:50.583994 5820 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 03:32:50.600127 5694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:50.600363 5694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:50.600405 5694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:50.601645 5694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:50.601711 5694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:50.601758 5694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:32:50.603567 5694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.129.keytab
--principal=oryx/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:39613
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=36057
--tserver_master_addrs=127.27.40.190:40413,127.27.40.189:44055,127.27.40.188:44133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:50.603852 5694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:50.604111 5694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:50.610337 5829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:50.610402 5828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:50.610383 5831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:50.610884 5694 server_base.cc:1034] running on GCE node
I20260809 03:32:50.611140 5694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:50.611418 5694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:50.612612 5694 hybrid_clock.cc:648] HybridClock initialized: now 1786246370612592 us; error 28 us; skew 500 ppm
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:50.615774 5694 init.cc:377] Logged in from keytab as oryx/127.27.40.129@KRBTEST.COM (short username oryx)
I20260809 03:32:50.616302 5694 webserver.cc:460] Webserver started at http://127.27.40.129:36057/ using document root <none> and password file <none>
I20260809 03:32:50.616567 5694 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:50.616636 5694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:50.618211 5694 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 03:32:50.618845 5843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:50.619048 5694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:32:50.619135 5694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "6694ec3787544426b332b9535dbe9ff3"
format_stamp: "Formatted at 2026-08-09 03:32:49 on dist-test-slave-gkw7"
server_key: "4ee146ea83d839a30710409fb7707267"
server_key_iv: "a95d872f62718cfa0542c36d5568dede"
server_key_version: "encryptionkey@0"
I20260809 03:32:50.619494 5694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:50.632583 5694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:50.634536 5694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:50.634662 5694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:50.634889 5694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:50.635210 5694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:50.635243 5694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:50.635265 5694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:50.635310 5694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:50.645895 5694 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:39613
I20260809 03:32:50.646008 6020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:39613 every 8 connection(s)
I20260809 03:32:50.646430 5694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:32:50.652732 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 5694
I20260809 03:32:50.652889 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 4902
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.188@KRBTEST.COM
I20260809 03:32:50.662775 6026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.647241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:44133 (local address 127.27.40.129:33625)
0809 03:32:50.647551 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.647598 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.649166 (+ 1568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.649467 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.649468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.650042 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.650402 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.650404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.651769 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.651770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.652701 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.652703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.653214 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.653215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.653420 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.655578 (+ 2158us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.655589 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.657374 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.659638 (+ 2264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.659645 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.659647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.659944 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.660145 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.660146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.660146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.661750 (+ 1604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.662104 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.662110 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.662265 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.662358 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.662526 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":4,"thread_start_us":143,"threads_started":1}
I20260809 03:32:50.663674 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.647630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:33625 (local address 127.27.40.188:44133)
0809 03:32:50.647974 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.647976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.647986 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:32:50.649289 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.649290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.649305 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.649428 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.650508 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.651645 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.652767 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.653093 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.653151 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.657480 (+ 4329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.657481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.657481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.657489 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.659550 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.660019 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.660020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.660021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.660085 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.661838 (+ 1753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.661839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.661839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.661983 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.662079 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.663449 (+ 1370us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.663556 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":86,"threads_started":1}
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.189@KRBTEST.COM
I20260809 03:32:50.664788 6021 heartbeater.cc:347] Connected to a master server at 127.27.40.188:44133
I20260809 03:32:50.664919 6021 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:50.665196 6021 heartbeater.cc:511] Master 127.27.40.188:44133 requested a full tablet report, sending...
I20260809 03:32:50.665681 5754 ts_manager.cc:194] Registered new tserver with Master: 6694ec3787544426b332b9535dbe9ff3 (127.27.40.129:39613)
I20260809 03:32:50.668395 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.647704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:43945 (local address 127.27.40.189:44055)
0809 03:32:50.647759 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.647761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.647868 (+ 107us) server_negotiation.cc:407] Connection header received
0809 03:32:50.649284 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.649286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.649302 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.649336 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.650495 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.651082 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.662365 (+ 11283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.662676 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.662724 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.665353 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.665355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.665355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.665364 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.667249 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.667621 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.667622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.667622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.667669 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.667846 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.667847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.667847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.667987 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.668023 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.668202 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.668275 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 03:32:50.668936 6028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.647625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:44055 (local address 127.27.40.129:43945)
0809 03:32:50.647780 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.647790 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.649145 (+ 1355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.649496 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.649497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.650030 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.650389 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.650394 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.651277 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.651278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.652432 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.652435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.662805 (+ 10370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.662806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.662885 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.663567 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.663572 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.665186 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.667339 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.667340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.667340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.667551 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.667727 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.667728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.667728 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.667775 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.668069 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.668070 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.668107 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.668658 (+ 551us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.668763 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":3,"thread_start_us":81,"threads_started":1}
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM
I20260809 03:32:50.675211 6023 heartbeater.cc:347] Connected to a master server at 127.27.40.189:44055
I20260809 03:32:50.675263 6023 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:50.675344 6023 heartbeater.cc:511] Master 127.27.40.189:44055 requested a full tablet report, sending...
I20260809 03:32:50.675825 5479 ts_manager.cc:194] Registered new tserver with Master: 6694ec3787544426b332b9535dbe9ff3 (127.27.40.129:39613)
I20260809 03:32:50.676338 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.647229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:51055 (local address 127.27.40.190:40413)
0809 03:32:50.647292 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.647295 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.647866 (+ 571us) server_negotiation.cc:407] Connection header received
0809 03:32:50.649220 (+ 1354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.649221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.649236 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.649286 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.650504 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.651239 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.652638 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.652986 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.653066 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.672907 (+ 19841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.672910 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.672910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.672921 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.675018 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.675482 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.675484 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.675485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.675550 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.675773 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.675774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.675775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.675927 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.675973 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.676202 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.676276 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 03:32:50.676498 6027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.647241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.129:51055)
0809 03:32:50.647725 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.647736 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.649095 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.649371 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.649375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.650035 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.650398 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.650400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.651330 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.651331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.652525 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.652527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.670223 (+ 17696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.670225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.670405 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.671224 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.671230 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.672785 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.675113 (+ 2328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.675115 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.675116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.675382 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.675626 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.675641 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.675642 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.675703 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.676019 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.676020 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.676078 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.676140 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.676262 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":6,"thread_start_us":97,"threads_started":1}
I20260809 03:32:50.677771 6022 heartbeater.cc:347] Connected to a master server at 127.27.40.190:40413
I20260809 03:32:50.677830 6022 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:50.677912 6022 heartbeater.cc:511] Master 127.27.40.190:40413 requested a full tablet report, sending...
I20260809 03:32:50.678269 4610 ts_manager.cc:194] Re-registered known tserver with Master: 6694ec3787544426b332b9535dbe9ff3 (127.27.40.129:39613)
I20260809 03:32:50.678804 4610 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:51055
I20260809 03:32:50.680493 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:44947
--local_ip_for_outbound_sockets=127.27.40.130
--tserver_master_addrs=127.27.40.190:40413
--webserver_port=40833
--webserver_interface=127.27.40.130
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.130.keytab
--principal=oryx/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.40.190:40413,127.27.40.189:44055
--tserver_master_addrs=127.27.40.190:40413,127.27.40.189:44055,127.27.40.188:44133 with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:50.767027 6031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:50.767242 6031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:50.767266 6031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:50.768410 6031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:50.768450 6031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:50.768476 6031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:32:50.770051 6031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.130.keytab
--principal=oryx/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:44947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=40833
--tserver_master_addrs=127.27.40.190:40413,127.27.40.189:44055,127.27.40.188:44133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:50.770308 6031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:50.770542 6031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:50.776588 6038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:50.776608 6041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:50.776996 6039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:50.777145 6031 server_base.cc:1034] running on GCE node
I20260809 03:32:50.777370 6031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:50.777673 6031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:50.780927 6031 hybrid_clock.cc:648] HybridClock initialized: now 1786246370780913 us; error 50 us; skew 500 ppm
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:50.783861 6031 init.cc:377] Logged in from keytab as oryx/127.27.40.130@KRBTEST.COM (short username oryx)
I20260809 03:32:50.784315 6031 webserver.cc:460] Webserver started at http://127.27.40.130:40833/ using document root <none> and password file <none>
I20260809 03:32:50.784556 6031 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:50.784646 6031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:50.786160 6031 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:32:50.786937 6057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:50.787140 6031 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:50.787254 6031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "f20d19f3ea0c469f95ba10c2417139bb"
format_stamp: "Formatted at 2026-08-09 03:32:49 on dist-test-slave-gkw7"
server_key: "7a31e60b1e9ba4cf58bfde5dc0478dcb"
server_key_iv: "351dfe0aa3cb0227e2f1d7d69974ca52"
server_key_version: "encryptionkey@0"
I20260809 03:32:50.787619 6031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:50.799598 6031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:50.802096 6031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:50.802266 6031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:50.802520 6031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:50.802847 6031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:50.802904 6031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:50.802951 6031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:50.802975 6031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:50.809208 6031 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:44947
I20260809 03:32:50.809294 6217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:44947 every 8 connection(s)
I20260809 03:32:50.809715 6031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:32:50.816484 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 6031
I20260809 03:32:50.816670 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 5087
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.189@KRBTEST.COM
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.188@KRBTEST.COM
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM
I20260809 03:32:50.825146 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.810892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:56759 (local address 127.27.40.189:44055)
0809 03:32:50.810941 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.810943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.811014 (+ 71us) server_negotiation.cc:407] Connection header received
0809 03:32:50.812223 (+ 1209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.812225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.812238 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.812272 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.813157 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.813726 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.814788 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.814992 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.815081 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.819074 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.819075 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.819076 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.819084 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.823362 (+ 4278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.823973 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.823974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.823974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.824030 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.824337 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.824338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.824338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.824491 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.824538 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.824950 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.825045 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 03:32:50.825281 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.810969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:34701 (local address 127.27.40.190:40413)
0809 03:32:50.811016 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.811018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.811029 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:32:50.812345 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.812347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.812361 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.812417 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.813167 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.813956 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.815132 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.815377 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.815409 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.819415 (+ 4006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.819417 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.819417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.819424 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.823879 (+ 4455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.824426 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.824427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.824427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.824482 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.824692 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.824693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.824694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.824822 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.824868 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.825084 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.825176 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 03:32:50.825812 6226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.810470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.130:34701)
0809 03:32:50.810669 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.810720 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.812047 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.812435 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.812435 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.812816 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.813037 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.813044 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.814075 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.814076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.815002 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.815004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.815520 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.815521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.815586 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.815982 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.815987 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.819320 (+ 3333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.824111 (+ 4791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.824112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.824113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.824342 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.824560 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.824561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.824561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.824619 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.824917 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.824918 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.824951 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.825387 (+ 436us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.825609 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":230,"thread_start_us":84,"threads_started":1}
I20260809 03:32:50.825814 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.810825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:44055 (local address 127.27.40.130:56759)
0809 03:32:50.810956 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.810965 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.812065 (+ 1100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.812366 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.812367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.812791 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.813037 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.813042 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.813831 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.813832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.814672 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.814674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.815093 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.815094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.815237 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.815836 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.815844 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.818994 (+ 3150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.823469 (+ 4475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.823479 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.823481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.823878 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.824096 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.824097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.824097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.824248 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.824587 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.824595 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.824788 (+ 193us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.825285 (+ 497us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.825532 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":46,"thread_start_us":75,"threads_started":1}
I20260809 03:32:50.826352 6223 heartbeater.cc:347] Connected to a master server at 127.27.40.189:44055
I20260809 03:32:50.826431 6223 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:50.826581 6223 heartbeater.cc:511] Master 127.27.40.189:44055 requested a full tablet report, sending...
I20260809 03:32:50.827077 5479 ts_manager.cc:194] Registered new tserver with Master: f20d19f3ea0c469f95ba10c2417139bb (127.27.40.130:44947)
I20260809 03:32:50.829201 6221 heartbeater.cc:347] Connected to a master server at 127.27.40.190:40413
I20260809 03:32:50.829252 6221 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:50.829304 6221 heartbeater.cc:511] Master 127.27.40.190:40413 requested a full tablet report, sending...
I20260809 03:32:50.829614 4610 ts_manager.cc:194] Re-registered known tserver with Master: f20d19f3ea0c469f95ba10c2417139bb (127.27.40.130:44947)
I20260809 03:32:50.830076 4610 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:34701
I20260809 03:32:50.838142 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.810469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:36007 (local address 127.27.40.188:44133)
0809 03:32:50.810519 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.810521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.810813 (+ 292us) server_negotiation.cc:407] Connection header received
0809 03:32:50.812202 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.812204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.812223 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.812282 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.813160 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.813956 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.815194 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.815511 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.815605 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.820771 (+ 5166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.820772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.820773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.820782 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.836770 (+ 15988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.837179 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.837181 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.837181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.837256 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.837472 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.837473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.837473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.837671 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.837747 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.837949 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.838044 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 03:32:50.838171 6225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.810470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:44133 (local address 127.27.40.130:36007)
0809 03:32:50.810673 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.810720 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.812047 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.812361 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.812365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.812809 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.813038 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.813044 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.814075 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.814076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.815112 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.815114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.815607 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.815608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.815688 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.816170 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.816175 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.820666 (+ 4491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.836878 (+ 16212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.836879 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.836880 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.837076 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.837329 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.837330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.837331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.837399 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.837752 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.837753 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.837795 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.837903 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.838038 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":60,"thread_start_us":96,"threads_started":1}
I20260809 03:32:50.838482 6220 heartbeater.cc:347] Connected to a master server at 127.27.40.188:44133
I20260809 03:32:50.838538 6220 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:50.838629 6220 heartbeater.cc:511] Master 127.27.40.188:44133 requested a full tablet report, sending...
I20260809 03:32:50.838959 5754 ts_manager.cc:194] Registered new tserver with Master: f20d19f3ea0c469f95ba10c2417139bb (127.27.40.130:44947)
I20260809 03:32:50.839920 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:36575
--local_ip_for_outbound_sockets=127.27.40.131
--tserver_master_addrs=127.27.40.190:40413
--webserver_port=43229
--webserver_interface=127.27.40.131
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.131.keytab
--principal=oryx/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.40.190:40413,127.27.40.189:44055
--tserver_master_addrs=127.27.40.190:40413,127.27.40.189:44055,127.27.40.188:44133 with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:32:50.924789 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.921846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:44133 (local address 127.0.0.1:36158)
0809 03:32:50.921911 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.921927 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.922000 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.922282 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.922283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.922285 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:50.922344 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.922347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.923068 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.923070 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.923921 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.923924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.924532 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.924533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.924592 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.924595 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.924666 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.924713 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 03:32:50.924789 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.921925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36158 (local address 127.27.40.188:44133)
0809 03:32:50.922004 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.922006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.922027 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:32:50.922060 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.922060 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.922116 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.922205 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:50.922404 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.922972 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.924027 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.924440 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.924528 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.924605 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.924672 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.924713 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:32:50.925153 5766 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 03:32:50.925330 5806 sys_catalog.cc:455] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b478a01c8d24855948ebf23178e3c8f. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } } }
I20260809 03:32:50.925441 5806 sys_catalog.cc:458] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: This master's current role is: LEARNER
W20260809 03:32:50.925388 6230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:50.925686 6230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:50.925715 6230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:50.926841 6230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:50.926877 6230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:50.926903 6230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:32:50.928339 6230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.131.keytab
--principal=oryx/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:36575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=43229
--tserver_master_addrs=127.27.40.190:40413,127.27.40.189:44055,127.27.40.188:44133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:50.928522 6230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:50.928709 6230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:50.933727 6241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:50.933784 6238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:50.933727 6237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:50.934010 6230 server_base.cc:1034] running on GCE node
I20260809 03:32:50.934250 6230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:50.934490 6230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:50.935652 6230 hybrid_clock.cc:648] HybridClock initialized: now 1786246370935634 us; error 38 us; skew 500 ppm
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:50.938225 6230 init.cc:377] Logged in from keytab as oryx/127.27.40.131@KRBTEST.COM (short username oryx)
I20260809 03:32:50.938666 6230 webserver.cc:460] Webserver started at http://127.27.40.131:43229/ using document root <none> and password file <none>
I20260809 03:32:50.938894 6230 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:50.938977 6230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:50.940305 6230 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:50.940868 6251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:50.941043 6230 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:50.941155 6230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3"
format_stamp: "Formatted at 2026-08-09 03:32:49 on dist-test-slave-gkw7"
server_key: "619493b72c0c78731041d3da49cb55b2"
server_key_iv: "6d6ca5e3af23014efcbc73d93bb50143"
server_key_version: "encryptionkey@0"
I20260809 03:32:50.941488 6230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:50.953922 6230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:50.955893 6230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:50.956058 6230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:50.956290 6230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:32:50.956612 6230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:50.956671 6230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:50.956712 6230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:50.956741 6230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:50.963140 6230 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:36575
I20260809 03:32:50.963181 6425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:36575 every 8 connection(s)
I20260809 03:32:50.963546 6230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:32:50.965626 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 6230
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.189@KRBTEST.COM
Aug 09 03:32:50 dist-test-slave-gkw7 krb5kdc[4540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246370, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.188@KRBTEST.COM
I20260809 03:32:50.975852 6445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.967068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36162 (local address 127.27.40.188:44133)
0809 03:32:50.967191 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.967193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.967199 (+ 6us) server_negotiation.cc:407] Connection header received
0809 03:32:50.967223 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.967224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.967239 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.967421 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.968430 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.969185 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.970278 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.970584 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.970643 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.972329 (+ 1686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.972331 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.972331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.972338 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.974471 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.974885 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.974886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.974887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.974949 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.975178 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.975179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.975179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.975326 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.975400 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.975622 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.975717 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":107,"thread_start_us":58,"threads_started":1}
I20260809 03:32:50.979235 6431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.964325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:44055 (local address 127.27.40.131:37739)
0809 03:32:50.964607 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.964619 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.965678 (+ 1059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.966044 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.966049 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.967340 (+ 1291us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.967804 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.967805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.968689 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.968689 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.970132 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.970135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.970689 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.970690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.970839 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.972038 (+ 1199us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.972044 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.975498 (+ 3454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.977419 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.977428 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.977431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.977758 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.977981 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.977981 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.977982 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.978133 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.978455 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.978463 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.978636 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.978750 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.978934 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":386,"thread_start_us":92,"threads_started":1}
I20260809 03:32:50.979614 6446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.967311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54126 (local address 127.27.40.189:44055)
0809 03:32:50.967421 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.967423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.967431 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:32:50.967453 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.967454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.967464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.967528 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.972151 (+ 4623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.972898 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.973911 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.974248 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.974317 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.975872 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.975873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.975873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.975880 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.978158 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.978562 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.978563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.978564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.978626 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.978841 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.978842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.978842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.978974 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.979071 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.979422 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.979503 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":61,"threads_started":1}
I20260809 03:32:50.979905 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.964369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:37739 (local address 127.27.40.189:44055)
0809 03:32:50.964439 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.964442 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.964709 (+ 267us) server_negotiation.cc:407] Connection header received
0809 03:32:50.965802 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.965804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.965817 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.965853 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.967882 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.968598 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.970235 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.970536 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.970579 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.975599 (+ 5020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.975600 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.975601 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.975608 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.977329 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.977854 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.977855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.977855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.977913 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.978217 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.978218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.978219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.978361 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.978417 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.979682 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.979761 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 03:32:50.980208 6428 heartbeater.cc:347] Connected to a master server at 127.27.40.189:44055
I20260809 03:32:50.980352 6428 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:50.980568 6428 heartbeater.cc:511] Master 127.27.40.189:44055 requested a full tablet report, sending...
I20260809 03:32:50.981123 5479 ts_manager.cc:194] Registered new tserver with Master: b8c3f42a3e8a45ee9806e74efc6ef5f3 (127.27.40.131:36575)
I20260809 03:32:50.981891 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.964443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:37169 (local address 127.27.40.188:44133)
0809 03:32:50.964488 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.964490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.964687 (+ 197us) server_negotiation.cc:407] Connection header received
0809 03:32:50.965816 (+ 1129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.965818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.965838 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.965875 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.967744 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.968538 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.970229 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.970538 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.970623 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.976843 (+ 6220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.976844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.976845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.976853 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.979190 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.980661 (+ 1471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.980662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.980663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.980726 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.981015 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.981016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.981017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.981175 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.981247 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.981648 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.981736 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":22}
I20260809 03:32:50.981906 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.964325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:44133 (local address 127.27.40.131:37169)
0809 03:32:50.964561 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.964604 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.965681 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.966045 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.966049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.967324 (+ 1275us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.967664 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.967677 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.968684 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.968685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.970132 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.970135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.970687 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.970688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.970839 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.971672 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.971678 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.975213 (+ 3535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.979276 (+ 4063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.979277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.979277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.979495 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.980842 (+ 1347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.980843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.980844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.980939 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.981381 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.981382 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.981428 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.981491 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.981695 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":106,"thread_start_us":169,"threads_started":1}
I20260809 03:32:50.982292 6426 heartbeater.cc:347] Connected to a master server at 127.27.40.188:44133
I20260809 03:32:50.982344 6426 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:50.982416 6426 heartbeater.cc:511] Master 127.27.40.188:44133 requested a full tablet report, sending...
I20260809 03:32:50.982704 6432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.964650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.131:42113)
0809 03:32:50.964778 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.964787 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.965679 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.966044 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.966049 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.967369 (+ 1320us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:50.967662 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.967668 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.969347 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.969348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.970193 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.970196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.970715 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.970716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.970839 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.971512 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:50.971522 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:50.973636 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:50.981375 (+ 7739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.981376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.981377 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.981734 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.982044 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:50.982045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:50.982046 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:50.982121 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:50.982400 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:50.982401 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:50.982443 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.982497 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.982618 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":32,"thread_start_us":63,"threads_started":1}
I20260809 03:32:50.982717 5754 ts_manager.cc:194] Registered new tserver with Master: b8c3f42a3e8a45ee9806e74efc6ef5f3 (127.27.40.131:36575)
I20260809 03:32:50.983047 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.964739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:42113 (local address 127.27.40.190:40413)
0809 03:32:50.964788 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.964790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.964832 (+ 42us) server_negotiation.cc:407] Connection header received
0809 03:32:50.965878 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.965880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.965892 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.965924 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.968038 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.968831 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.970278 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.970578 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.970662 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.973735 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.973737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.973737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.973745 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.981274 (+ 7529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.981837 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.981839 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.981839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.981941 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.982199 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.982200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.982200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.982323 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.982376 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.982867 (+ 491us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.982951 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 03:32:50.983359 6427 heartbeater.cc:347] Connected to a master server at 127.27.40.190:40413
I20260809 03:32:50.983410 6427 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:50.983456 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.966806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51088 (local address 127.27.40.190:40413)
0809 03:32:50.967502 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.967504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.967511 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:32:50.967554 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.967554 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.967584 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.971035 (+ 3451us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:50.971068 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.976809 (+ 5741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.977854 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.978208 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.978514 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.979603 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:50.979605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:50.979605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:50.979613 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:50.981794 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.982274 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.982275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.982275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.982378 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:50.982619 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:50.982620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:50.982620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:50.982743 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:50.983216 (+ 473us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.983258 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.983353 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":131,"threads_started":1}
I20260809 03:32:50.983481 6427 heartbeater.cc:511] Master 127.27.40.190:40413 requested a full tablet report, sending...
I20260809 03:32:50.983763 4609 ts_manager.cc:194] Re-registered known tserver with Master: b8c3f42a3e8a45ee9806e74efc6ef5f3 (127.27.40.131:36575)
I20260809 03:32:50.984297 4609 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:42113
I20260809 03:32:50.985001 4609 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51088:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:32:50.986676 4609 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:50.987494 5710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: NON_VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: true } } }
I20260809 03:32:50.987593 5710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:50.988346 5710 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f: attempting to promote NON_VOTER b505344d51174a9c9651e17bb1061ae0 to VOTER
I20260809 03:32:50.988520 5710 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } }
I20260809 03:32:50.989002 5490 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 FOLLOWER]: Refusing update from remote peer 0b478a01c8d24855948ebf23178e3c8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 03:32:50.989034 5766 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 LEARNER]: Refusing update from remote peer 0b478a01c8d24855948ebf23178e3c8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 03:32:50.989202 5419 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 03:32:50.989303 5417 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 03:32:50.989948 5419 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b505344d51174a9c9651e17bb1061ae0 (127.27.40.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } } }
I20260809 03:32:50.990281 5766 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b505344d51174a9c9651e17bb1061ae0 (127.27.40.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } } }
I20260809 03:32:50.991019 5419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } } }
I20260809 03:32:50.991078 5710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } } }
I20260809 03:32:50.991173 5710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:50.991236 5710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } } }
I20260809 03:32:50.991321 5710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:50.991276 5490 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b505344d51174a9c9651e17bb1061ae0 (127.27.40.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } } }
I20260809 03:32:50.991451 5806 sys_catalog.cc:455] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } } }
I20260809 03:32:50.991540 5806 sys_catalog.cc:458] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:32:50.991096 5419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:50.992105 5708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b478a01c8d24855948ebf23178e3c8f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } } }
I20260809 03:32:50.992225 5708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:32:50.998479 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.994265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:36575 (local address 127.0.0.1:59484)
0809 03:32:50.994441 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.994456 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.994508 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.995170 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.995171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.995173 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:50.995219 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.995222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.996180 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.996181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.997147 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.997150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.998188 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.998189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.998253 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.998258 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.998327 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.998401 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260809 03:32:50.998687 6459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.994452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59484 (local address 127.27.40.131:36575)
0809 03:32:50.994935 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.994940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.994951 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:32:50.994993 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.994995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.995019 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.995135 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:50.995289 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.996083 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.997251 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.998113 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.998182 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.998379 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.998412 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.998457 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":112,"threads_started":1}
I20260809 03:32:50.999636 6458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.994534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:44947 (local address 127.0.0.1:55118)
0809 03:32:50.995238 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.995250 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.995304 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.995612 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.995613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.995615 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:50.995657 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.995659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.996926 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.996926 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.997860 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.997863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.999389 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.999390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.999451 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.999457 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.999518 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.999556 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":3,"thread_start_us":102,"threads_started":1}
I20260809 03:32:50.999660 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.994662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38290 (local address 127.27.40.129:39613)
0809 03:32:50.994889 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.994894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.994906 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:32:50.994961 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.994963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.994981 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.995070 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:50.995302 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.996611 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.998085 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.998822 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.998913 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.998982 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.999292 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 03:32:50.999370 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":85,"threads_started":1}
I20260809 03:32:50.999706 6319 tablet_service.cc:1467] Processing CreateTablet for tablet 76a536d1b575489e86a89562a491c240 (DEFAULT_TABLE table=test-table [id=7684b6f6baec406c95a27f5648f911c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:51.000152 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.994780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55118 (local address 127.27.40.130:44947)
0809 03:32:50.995223 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:50.995228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:50.995374 (+ 146us) server_negotiation.cc:407] Connection header received
0809 03:32:50.995430 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:50.995432 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:50.995449 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:50.995523 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:50.995737 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.996856 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.997949 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.999068 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.999156 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.999350 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:50.999960 (+ 610us) server_negotiation.cc:299] Negotiation successful
0809 03:32:51.000007 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":86,"threads_started":1}
I20260809 03:32:51.000178 6319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76a536d1b575489e86a89562a491c240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:50.999864 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:50.994384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:39613 (local address 127.0.0.1:38290)
0809 03:32:50.994687 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:50.994698 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:50.994752 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:50.995101 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:50.995102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:50.995106 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:50.995185 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.995188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.996720 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.996721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.997988 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:50.997992 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:50.999067 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:50.999068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:50.999157 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:50.999161 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:50.999751 (+ 590us) client_negotiation.cc:240] Negotiation successful
0809 03:32:50.999808 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":28,"thread_start_us":101,"threads_started":1}
I20260809 03:32:51.001000 6130 tablet_service.cc:1467] Processing CreateTablet for tablet 76a536d1b575489e86a89562a491c240 (DEFAULT_TABLE table=test-table [id=7684b6f6baec406c95a27f5648f911c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:51.001135 5923 tablet_service.cc:1467] Processing CreateTablet for tablet 76a536d1b575489e86a89562a491c240 (DEFAULT_TABLE table=test-table [id=7684b6f6baec406c95a27f5648f911c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:51.001411 6130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76a536d1b575489e86a89562a491c240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:51.001679 5923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76a536d1b575489e86a89562a491c240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:51.005963 6466 tablet_bootstrap.cc:492] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3: Bootstrap starting.
I20260809 03:32:51.006239 6465 tablet_bootstrap.cc:492] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3: Bootstrap starting.
I20260809 03:32:51.006655 6466 tablet_bootstrap.cc:654] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:51.007328 6466 log.cc:824] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:51.007603 6465 tablet_bootstrap.cc:654] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:51.007557 6467 tablet_bootstrap.cc:492] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb: Bootstrap starting.
I20260809 03:32:51.008482 6465 log.cc:824] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:51.008836 6466 tablet_bootstrap.cc:492] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3: No bootstrap required, opened a new log
I20260809 03:32:51.008919 6466 ts_tablet_manager.cc:1397] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:32:51.009259 6467 tablet_bootstrap.cc:654] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:51.009723 6465 tablet_bootstrap.cc:492] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3: No bootstrap required, opened a new log
I20260809 03:32:51.009799 6465 ts_tablet_manager.cc:1397] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 03:32:51.010238 6467 log.cc:824] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:51.011565 6467 tablet_bootstrap.cc:492] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb: No bootstrap required, opened a new log
I20260809 03:32:51.011672 6467 ts_tablet_manager.cc:1397] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260809 03:32:51.013090 6465 raft_consensus.cc:348] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:51.013264 6465 raft_consensus.cc:374] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:51.013307 6465 raft_consensus.cc:729] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c3f42a3e8a45ee9806e74efc6ef5f3, State: Initialized, Role: FOLLOWER
I20260809 03:32:51.013427 6465 consensus_queue.cc:260] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:51.013854 6465 ts_tablet_manager.cc:1428] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 03:32:51.013927 6427 heartbeater.cc:503] Master 127.27.40.190:40413 was elected leader, sending a full tablet report...
I20260809 03:32:51.014214 6466 raft_consensus.cc:348] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:51.015764 6466 raft_consensus.cc:374] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:51.015859 6466 raft_consensus.cc:729] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6694ec3787544426b332b9535dbe9ff3, State: Initialized, Role: FOLLOWER
I20260809 03:32:51.016000 6466 consensus_queue.cc:260] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:51.016383 6466 ts_tablet_manager.cc:1428] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 03:32:51.016273 6467 raft_consensus.cc:348] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:51.016556 6022 heartbeater.cc:503] Master 127.27.40.190:40413 was elected leader, sending a full tablet report...
I20260809 03:32:51.016564 6467 raft_consensus.cc:374] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:51.016610 6467 raft_consensus.cc:729] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f20d19f3ea0c469f95ba10c2417139bb, State: Initialized, Role: FOLLOWER
I20260809 03:32:51.016786 6467 consensus_queue.cc:260] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:51.017181 6467 ts_tablet_manager.cc:1428] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260809 03:32:51.017343 6221 heartbeater.cc:503] Master 127.27.40.190:40413 was elected leader, sending a full tablet report...
W20260809 03:32:51.061379 6224 tablet.cc:2369] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:32:51.147859 6024 tablet.cc:2369] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:32:51.173380 6473 raft_consensus.cc:482] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:51.173588 6473 raft_consensus.cc:504] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:51.174131 6473 leader_election.cc:288] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6694ec3787544426b332b9535dbe9ff3 (127.27.40.129:39613), f20d19f3ea0c469f95ba10c2417139bb (127.27.40.130:44947)
I20260809 03:32:51.178231 6432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:51.174361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:44947 (local address 127.27.40.131:51207)
0809 03:32:51.174431 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:51.174445 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:51.174514 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:51.174679 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:51.174680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:51.174682 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:51.174868 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:51.174871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:51.175784 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:51.175785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:51.177125 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:51.177128 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:51.177927 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:51.177928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:51.178066 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:51.178071 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:32:51.178129 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 03:32:51.178178 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":25}
I20260809 03:32:51.178335 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:51.174489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:51207 (local address 127.27.40.130:44947)
0809 03:32:51.174535 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:51.174537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:51.174547 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:32:51.174578 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:51.174579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:51.174590 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:51.174663 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:51.174989 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:51.175669 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:51.177229 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:51.177840 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:51.177946 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:51.177958 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:51.178142 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 03:32:51.178187 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 03:32:51.178524 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:51.174472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:55099 (local address 127.27.40.129:39613)
0809 03:32:51.174538 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:51.174540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:51.174643 (+ 103us) server_negotiation.cc:407] Connection header received
0809 03:32:51.174716 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:51.174717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:51.174734 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:51.174801 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:51.174955 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:51.175611 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:51.177580 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:51.177991 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:51.178102 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:51.178112 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:51.178414 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 03:32:51.178463 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 03:32:51.178603 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:51.174361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:39613 (local address 127.27.40.131:55099)
0809 03:32:51.174563 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:51.174573 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:51.174659 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:51.174804 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:51.174805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:51.174806 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:51.174895 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:51.174898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:51.175709 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:51.175710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:51.177473 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:51.177476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:51.178093 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:51.178094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:51.178321 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:51.178325 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:51.178394 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 03:32:51.178442 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":209}
I20260809 03:32:51.178833 6160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76a536d1b575489e86a89562a491c240" candidate_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f20d19f3ea0c469f95ba10c2417139bb" is_pre_election: true
I20260809 03:32:51.179032 6160 raft_consensus.cc:2454] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c3f42a3e8a45ee9806e74efc6ef5f3 in term 0.
I20260809 03:32:51.179050 5951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76a536d1b575489e86a89562a491c240" candidate_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6694ec3787544426b332b9535dbe9ff3" is_pre_election: true
I20260809 03:32:51.179212 5951 raft_consensus.cc:2454] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c3f42a3e8a45ee9806e74efc6ef5f3 in term 0.
I20260809 03:32:51.179356 6254 leader_election.cc:302] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8c3f42a3e8a45ee9806e74efc6ef5f3, f20d19f3ea0c469f95ba10c2417139bb; no voters:
I20260809 03:32:51.179484 6473 raft_consensus.cc:2781] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:51.179563 6473 raft_consensus.cc:482] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:51.179600 6473 raft_consensus.cc:3037] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:51.180749 6473 raft_consensus.cc:504] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:51.180928 6473 leader_election.cc:288] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [CANDIDATE]: Term 1 election: Requested vote from peers 6694ec3787544426b332b9535dbe9ff3 (127.27.40.129:39613), f20d19f3ea0c469f95ba10c2417139bb (127.27.40.130:44947)
I20260809 03:32:51.181070 6160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76a536d1b575489e86a89562a491c240" candidate_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f20d19f3ea0c469f95ba10c2417139bb"
I20260809 03:32:51.181057 5951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76a536d1b575489e86a89562a491c240" candidate_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6694ec3787544426b332b9535dbe9ff3"
I20260809 03:32:51.181155 6160 raft_consensus.cc:3037] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:51.181174 5951 raft_consensus.cc:3037] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:51.182336 5951 raft_consensus.cc:2454] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c3f42a3e8a45ee9806e74efc6ef5f3 in term 1.
I20260809 03:32:51.182330 6160 raft_consensus.cc:2454] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c3f42a3e8a45ee9806e74efc6ef5f3 in term 1.
I20260809 03:32:51.182525 6254 leader_election.cc:302] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8c3f42a3e8a45ee9806e74efc6ef5f3, f20d19f3ea0c469f95ba10c2417139bb; no voters:
I20260809 03:32:51.182660 6473 raft_consensus.cc:2781] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:51.182818 6473 raft_consensus.cc:686] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 1 LEADER]: Becoming Leader. State: Replica: b8c3f42a3e8a45ee9806e74efc6ef5f3, State: Running, Role: LEADER
I20260809 03:32:51.183036 6473 consensus_queue.cc:237] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:51.184028 4609 catalog_manager.cc:5495] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8c3f42a3e8a45ee9806e74efc6ef5f3 (127.27.40.131). New cstate: current_term: 1 leader_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } health_report { overall_health: UNKNOWN } } }
W20260809 03:32:51.214442 6429 tablet.cc:2369] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:32:51.225236 6459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:51.222610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59492 (local address 127.27.40.131:36575)
0809 03:32:51.222681 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:51.222684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:51.222693 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:51.222740 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:51.222741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:51.222760 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:51.222827 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:51.222993 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:51.223658 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:51.224401 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:51.224612 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:51.224697 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:51.224848 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:51.224955 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:51.225035 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:51.225155 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 03:32:51.225176 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 03:32:51.227401 5951 raft_consensus.cc:1261] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3 [term 1 FOLLOWER]: Refusing update from remote peer b8c3f42a3e8a45ee9806e74efc6ef5f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:51.227401 6160 raft_consensus.cc:1261] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb [term 1 FOLLOWER]: Refusing update from remote peer b8c3f42a3e8a45ee9806e74efc6ef5f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:51.227922 6477 consensus_queue.cc:1035] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:51.228082 6473 consensus_queue.cc:1035] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:51.231581 6485 mvcc.cc:205] Tried to move back new op lower bound from 7316465136543924224 to 7316465136366534656. Current Snapshot: MvccSnapshot[applied={T|T < 7316465136543924224}]
I20260809 03:32:51.232849 6484 mvcc.cc:205] Tried to move back new op lower bound from 7316465136543924224 to 7316465136366534656. Current Snapshot: MvccSnapshot[applied={T|T < 7316465136543924224 or (T in {7316465136543924224})}]
I20260809 03:32:51.234309 4609 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:51.234403 4609 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:51.235913 4609 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b478a01c8d24855948ebf23178e3c8f: Sending DeleteTablet for 3 replicas of tablet 76a536d1b575489e86a89562a491c240
I20260809 03:32:51.236327 5923 tablet_service.cc:1514] Processing DeleteTablet for tablet 76a536d1b575489e86a89562a491c240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:51 UTC) from {username='oryx', principal='oryx/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:38290
I20260809 03:32:51.236426 6319 tablet_service.cc:1514] Processing DeleteTablet for tablet 76a536d1b575489e86a89562a491c240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:51 UTC) from {username='oryx', principal='oryx/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:59484
I20260809 03:32:51.236720 6498 tablet_replica.cc:330] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3: stopping tablet replica
I20260809 03:32:51.236871 6498 raft_consensus.cc:2229] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:32:51.236958 6130 tablet_service.cc:1514] Processing DeleteTablet for tablet 76a536d1b575489e86a89562a491c240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:51 UTC) from {username='oryx', principal='oryx/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:55118
I20260809 03:32:51.237031 6498 raft_consensus.cc:2258] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:51.237946 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 4563
I20260809 03:32:51.238001 6498 ts_tablet_manager.cc:1905] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:51.245818 6497 tablet_replica.cc:330] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3: stopping tablet replica
I20260809 03:32:51.246071 6498 ts_tablet_manager.cc:1918] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:51.246177 6498 log.cc:1196] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/76a536d1b575489e86a89562a491c240
I20260809 03:32:51.246471 6498 ts_tablet_manager.cc:1939] T 76a536d1b575489e86a89562a491c240 P b8c3f42a3e8a45ee9806e74efc6ef5f3: Deleting consensus metadata
I20260809 03:32:51.247061 6497 raft_consensus.cc:2229] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:51.247242 6497 raft_consensus.cc:2258] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:51.248488 6497 ts_tablet_manager.cc:1905] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:51.250622 6497 ts_tablet_manager.cc:1918] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:51.250717 6497 log.cc:1196] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-0/wal/wals/76a536d1b575489e86a89562a491c240
I20260809 03:32:51.251009 6497 ts_tablet_manager.cc:1939] T 76a536d1b575489e86a89562a491c240 P 6694ec3787544426b332b9535dbe9ff3: Deleting consensus metadata
W20260809 03:32:51.254678 5845 connection.cc:535] server connection from 127.0.0.1:38290 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 03:32:51.255004 6257 connection.cc:535] server connection from 127.0.0.1:59484 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 03:32:51.255690 6500 tablet_replica.cc:330] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb: stopping tablet replica
I20260809 03:32:51.255822 6500 raft_consensus.cc:2229] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:51.255944 6500 raft_consensus.cc:2258] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:51.256896 6500 ts_tablet_manager.cc:1905] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:51.259346 6500 ts_tablet_manager.cc:1918] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:51.259433 6500 log.cc:1196] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/76a536d1b575489e86a89562a491c240
I20260809 03:32:51.259689 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:51.237395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36164 (local address 127.27.40.188:44133)
0809 03:32:51.237439 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:51.237441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:51.237450 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:51.237471 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:51.237471 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:51.237483 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:51.237569 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:51.249695 (+ 12126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:51.250788 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:51.252417 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:51.252774 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:51.252849 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:51.255177 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:51.255179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:51.255180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:51.255189 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:51.257689 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:51.258363 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:51.258364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:51.258365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:51.258435 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:51.258666 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:51.258667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:51.258667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:51.258807 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:51.258903 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:51.259484 (+ 581us) server_negotiation.cc:299] Negotiation successful
0809 03:32:51.259568 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 03:32:51.259742 6500 ts_tablet_manager.cc:1939] T 76a536d1b575489e86a89562a491c240 P f20d19f3ea0c469f95ba10c2417139bb: Deleting consensus metadata
W20260809 03:32:51.260089 6062 connection.cc:441] server connection from 127.0.0.1:55118 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55118 (request call id 1) could send its response
I20260809 03:32:51.266880 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:51.257891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54130 (local address 127.27.40.189:44055)
0809 03:32:51.257950 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:51.257952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:51.258344 (+ 392us) server_negotiation.cc:407] Connection header received
0809 03:32:51.258423 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:51.258424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:51.258437 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:51.258506 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:51.260747 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:51.261675 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:51.262405 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:51.262613 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:51.262692 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:51.264025 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:51.264026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:51.264026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:51.264033 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:51.265727 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:51.266095 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:51.266096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:51.266096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:51.266169 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:51.266367 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:51.266368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:51.266368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:51.266499 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:51.266612 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:51.266708 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 03:32:51.266777 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 03:32:51.267469 6445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:51.256592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36174 (local address 127.27.40.188:44133)
0809 03:32:51.256639 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:51.256641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:51.257244 (+ 603us) server_negotiation.cc:407] Connection header received
0809 03:32:51.257352 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:51.257353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:51.257366 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:51.257400 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:51.258881 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:51.259698 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:51.262203 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:51.262530 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:51.262576 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:51.264377 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:51.264378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:51.264379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:51.264387 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:51.266018 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:51.266455 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:51.266456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:51.266456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:51.266507 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:51.266707 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:51.266708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:51.266709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:51.266826 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:51.266890 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:51.267271 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 03:32:51.267359 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 03:32:52.233220 6513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:52.232859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.130:43557)
0809 03:32:52.233019 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:52.233125 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":78,"threads_started":1}
I20260809 03:32:52.233616 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:52.233361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.131:45483)
0809 03:32:52.233482 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:52.233580 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
W20260809 03:32:52.233618 6221 heartbeater.cc:644] Failed to heartbeat to 127.27.40.190:40413 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
W20260809 03:32:52.233860 6427 heartbeater.cc:644] Failed to heartbeat to 127.27.40.190:40413 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
I20260809 03:32:52.234515 6517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:52.234258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.129:48469)
0809 03:32:52.234379 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:52.234467 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
W20260809 03:32:52.234834 6022 heartbeater.cc:644] Failed to heartbeat to 127.27.40.190:40413 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
I20260809 03:32:52.736707 6518 raft_consensus.cc:482] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0b478a01c8d24855948ebf23178e3c8f)
I20260809 03:32:52.736845 6518 raft_consensus.cc:504] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } }
I20260809 03:32:52.737313 6518 leader_election.cc:288] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0b478a01c8d24855948ebf23178e3c8f (127.27.40.190:40413), 35d92e21b47a498796999adcfc6ba6fb (127.27.40.189:44055)
I20260809 03:32:52.737293 6520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0b478a01c8d24855948ebf23178e3c8f)
I20260809 03:32:52.737448 6520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } }
I20260809 03:32:52.737761 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:52.737479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51606)
0809 03:32:52.737630 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:52.737714 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":74,"threads_started":1}
I20260809 03:32:52.737759 6520 leader_election.cc:288] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0b478a01c8d24855948ebf23178e3c8f (127.27.40.190:40413), b505344d51174a9c9651e17bb1061ae0 (127.27.40.188:44133)
W20260809 03:32:52.738001 5732 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
I20260809 03:32:52.738243 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:52.737830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51622)
0809 03:32:52.738136 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:52.738197 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":54,"threads_started":1}
W20260809 03:32:52.738384 5454 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
I20260809 03:32:52.738843 6530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:52.738634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51636)
0809 03:32:52.738766 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:52.738795 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260809 03:32:52.738847 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:52.738714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51638)
0809 03:32:52.738769 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:52.738799 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 03:32:52.738929 5454 leader_election.cc:334] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0b478a01c8d24855948ebf23178e3c8f (127.27.40.190:40413): Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
W20260809 03:32:52.739010 5732 leader_election.cc:334] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0b478a01c8d24855948ebf23178e3c8f (127.27.40.190:40413): Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
I20260809 03:32:52.741144 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:52.737681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:44055 (local address 127.0.0.1:54146)
0809 03:32:52.737824 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:52.737836 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:52.737907 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:52.738106 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:52.738107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:52.738111 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:52.738173 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:52.738175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:52.739162 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:52.739163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:52.740256 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:52.740259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:52.740903 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:52.740904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:52.741008 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:52.741013 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:32:52.741056 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 03:32:52.741097 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":238,"threads_started":1}
I20260809 03:32:52.741185 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:52.737754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54146 (local address 127.27.40.189:44055)
0809 03:32:52.737914 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:52.737917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:52.737931 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:32:52.737962 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:52.737963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:52.737975 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:52.738046 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:52.738258 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:52.739066 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:52.740358 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:52.740820 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:52.740893 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:52.740905 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:52.741083 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 03:32:52.741123 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":56,"threads_started":1}
I20260809 03:32:52.741380 5490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b505344d51174a9c9651e17bb1061ae0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "35d92e21b47a498796999adcfc6ba6fb" is_pre_election: true
I20260809 03:32:52.741510 5490 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b505344d51174a9c9651e17bb1061ae0 in term 1.
I20260809 03:32:52.741636 6534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:52.737936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:44133 (local address 127.0.0.1:36182)
0809 03:32:52.738149 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:52.738160 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:52.738212 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:52.738645 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:52.738646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:52.738649 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:52.738710 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:52.738711 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:52.739592 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:52.739593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:52.740632 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:52.740635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:52.741376 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:52.741377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:52.741442 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:52.741446 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:52.741492 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 03:32:52.741567 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":52,"threads_started":1}
I20260809 03:32:52.741645 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:52.738113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36182 (local address 127.27.40.188:44133)
0809 03:32:52.738414 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:52.738417 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:52.738425 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:32:52.738484 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:52.738485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:52.738494 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:52.738821 (+ 327us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:52.738859 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:52.739503 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:52.740718 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:52.741258 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:52.741366 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:52.741377 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:52.741518 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 03:32:52.741583 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":51,"threads_started":1}
I20260809 03:32:52.741904 5766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "35d92e21b47a498796999adcfc6ba6fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b505344d51174a9c9651e17bb1061ae0" is_pre_election: true
I20260809 03:32:52.741987 5766 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35d92e21b47a498796999adcfc6ba6fb in term 1.
I20260809 03:32:52.742012 5732 leader_election.cc:302] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35d92e21b47a498796999adcfc6ba6fb, b505344d51174a9c9651e17bb1061ae0; no voters: 0b478a01c8d24855948ebf23178e3c8f
I20260809 03:32:52.742164 5453 leader_election.cc:302] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35d92e21b47a498796999adcfc6ba6fb, b505344d51174a9c9651e17bb1061ae0; no voters: 0b478a01c8d24855948ebf23178e3c8f
I20260809 03:32:52.742169 6518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 03:32:52.742228 6518 raft_consensus.cc:482] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0b478a01c8d24855948ebf23178e3c8f)
I20260809 03:32:52.742260 6518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:32:52.742300 6520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 03:32:52.742336 6520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0b478a01c8d24855948ebf23178e3c8f)
I20260809 03:32:52.742365 6520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:32:52.743016 6518 raft_consensus.cc:504] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } }
I20260809 03:32:52.743016 6520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } }
I20260809 03:32:52.743151 6518 leader_election.cc:288] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [CANDIDATE]: Term 2 election: Requested vote from peers 0b478a01c8d24855948ebf23178e3c8f (127.27.40.190:40413), 35d92e21b47a498796999adcfc6ba6fb (127.27.40.189:44055)
I20260809 03:32:52.743160 6520 leader_election.cc:288] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [CANDIDATE]: Term 2 election: Requested vote from peers 0b478a01c8d24855948ebf23178e3c8f (127.27.40.190:40413), b505344d51174a9c9651e17bb1061ae0 (127.27.40.188:44133)
I20260809 03:32:52.743315 5766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "35d92e21b47a498796999adcfc6ba6fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b505344d51174a9c9651e17bb1061ae0"
I20260809 03:32:52.743363 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:52.743237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51646)
0809 03:32:52.743300 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:52.743336 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 03:32:52.743359 5490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b505344d51174a9c9651e17bb1061ae0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "35d92e21b47a498796999adcfc6ba6fb"
I20260809 03:32:52.743404 5766 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 35d92e21b47a498796999adcfc6ba6fb in current term 2: Already voted for candidate b505344d51174a9c9651e17bb1061ae0 in this term.
I20260809 03:32:52.743435 5490 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b505344d51174a9c9651e17bb1061ae0 in current term 2: Already voted for candidate 35d92e21b47a498796999adcfc6ba6fb in this term.
I20260809 03:32:52.743547 6534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:52.743256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51656)
0809 03:32:52.743341 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:52.743362 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 03:32:52.743784 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:52.743696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51662)
0809 03:32:52.743728 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:52.743758 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 03:32:52.743819 6534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:52.743743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51666)
0809 03:32:52.743778 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:52.743798 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 03:32:52.743865 5454 leader_election.cc:334] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0b478a01c8d24855948ebf23178e3c8f (127.27.40.190:40413): Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
I20260809 03:32:52.743891 5454 leader_election.cc:302] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35d92e21b47a498796999adcfc6ba6fb; no voters: 0b478a01c8d24855948ebf23178e3c8f, b505344d51174a9c9651e17bb1061ae0
W20260809 03:32:52.743873 5732 leader_election.cc:334] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0b478a01c8d24855948ebf23178e3c8f (127.27.40.190:40413): Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
I20260809 03:32:52.743935 5732 leader_election.cc:302] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b505344d51174a9c9651e17bb1061ae0; no voters: 0b478a01c8d24855948ebf23178e3c8f, 35d92e21b47a498796999adcfc6ba6fb
I20260809 03:32:52.743979 6520 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 03:32:52.744012 6518 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 03:32:53.234287 6539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:53.233972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.130:35563)
0809 03:32:53.234180 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:53.234228 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":71,"threads_started":1}
I20260809 03:32:53.234373 6540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:53.234154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.131:40119)
0809 03:32:53.234271 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:53.234310 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
I20260809 03:32:53.235404 6541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:53.235197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.129:44905)
0809 03:32:53.235343 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:53.235368 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":64,"threads_started":1}
I20260809 03:32:54.235081 6542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.234773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.131:44865)
0809 03:32:54.234976 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.235017 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":74,"threads_started":1}
I20260809 03:32:54.235073 6543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.234773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.130:47931)
0809 03:32:54.234947 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.234992 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":81,"threads_started":1}
I20260809 03:32:54.236063 6547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.235844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.129:59729)
0809 03:32:54.235978 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.236016 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260809 03:32:54.765566 6548 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:54.765686 6548 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } }
I20260809 03:32:54.765885 6548 leader_election.cc:288] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0b478a01c8d24855948ebf23178e3c8f (127.27.40.190:40413), b505344d51174a9c9651e17bb1061ae0 (127.27.40.188:44133)
I20260809 03:32:54.766100 5766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "35d92e21b47a498796999adcfc6ba6fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b505344d51174a9c9651e17bb1061ae0" is_pre_election: true
I20260809 03:32:54.766248 5766 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35d92e21b47a498796999adcfc6ba6fb in term 2.
I20260809 03:32:54.766300 6549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.766027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51954)
0809 03:32:54.766190 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.766249 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":112,"threads_started":1}
I20260809 03:32:54.766443 5453 leader_election.cc:302] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35d92e21b47a498796999adcfc6ba6fb, b505344d51174a9c9651e17bb1061ae0; no voters:
I20260809 03:32:54.766721 6548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 03:32:54.766786 6548 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:54.766834 6548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 2 FOLLOWER]: Advancing to term 3
I20260809 03:32:54.766789 6549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.766689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51964)
0809 03:32:54.766737 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.766769 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 03:32:54.766945 5454 leader_election.cc:334] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0b478a01c8d24855948ebf23178e3c8f (127.27.40.190:40413): Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
I20260809 03:32:54.767642 6548 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } }
I20260809 03:32:54.767802 6548 leader_election.cc:288] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [CANDIDATE]: Term 3 election: Requested vote from peers 0b478a01c8d24855948ebf23178e3c8f (127.27.40.190:40413), b505344d51174a9c9651e17bb1061ae0 (127.27.40.188:44133)
I20260809 03:32:54.768002 6549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.767897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51980)
0809 03:32:54.767948 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.767977 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 03:32:54.767987 5766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "35d92e21b47a498796999adcfc6ba6fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b505344d51174a9c9651e17bb1061ae0"
I20260809 03:32:54.768062 5766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 2 FOLLOWER]: Advancing to term 3
I20260809 03:32:54.768352 6549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.768257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:51990)
0809 03:32:54.768304 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.768332 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 03:32:54.768443 5454 leader_election.cc:334] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0b478a01c8d24855948ebf23178e3c8f (127.27.40.190:40413): Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
I20260809 03:32:54.768929 5766 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35d92e21b47a498796999adcfc6ba6fb in term 3.
I20260809 03:32:54.769124 5453 leader_election.cc:302] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35d92e21b47a498796999adcfc6ba6fb, b505344d51174a9c9651e17bb1061ae0; no voters: 0b478a01c8d24855948ebf23178e3c8f
I20260809 03:32:54.769238 6548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 3 FOLLOWER]: Leader election won for term 3
I20260809 03:32:54.769351 6548 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [term 3 LEADER]: Becoming Leader. State: Replica: 35d92e21b47a498796999adcfc6ba6fb, State: Running, Role: LEADER
I20260809 03:32:54.769421 6548 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } }
I20260809 03:32:54.769860 6551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35d92e21b47a498796999adcfc6ba6fb. Latest consensus state: current_term: 3 leader_uuid: "35d92e21b47a498796999adcfc6ba6fb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } } }
I20260809 03:32:54.769977 6551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:54.770212 6553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:54.770633 6553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:54.770736 6553 catalog_manager.cc:1259] Loaded cluster ID: e78f5199826845eeb37908ef76813cbe
I20260809 03:32:54.770772 6553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:32:54.770982 6553 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:54.771093 6553 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: Loaded TSK: 0
I20260809 03:32:54.771256 6553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:32:54.852546 5479 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:32:54.854287 5479 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:54.855391 6549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.855232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:52000)
0809 03:32:54.855296 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.855355 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 03:32:54.855448 5766 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [term 3 FOLLOWER]: Refusing update from remote peer 35d92e21b47a498796999adcfc6ba6fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260809 03:32:54.855674 6551 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 03:32:54.855786 6549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.855677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.0.0.1:52010)
0809 03:32:54.855731 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.855761 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 03:32:54.855866 5454 consensus_peers.cc:487] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb -> Peer 0b478a01c8d24855948ebf23178e3c8f (127.27.40.190:40413): Couldn't send request to peer 0b478a01c8d24855948ebf23178e3c8f. Status: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 03:32:54.856366 6556 sys_catalog.cc:455] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35d92e21b47a498796999adcfc6ba6fb. Latest consensus state: current_term: 3 leader_uuid: "35d92e21b47a498796999adcfc6ba6fb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } } }
I20260809 03:32:54.856473 6556 sys_catalog.cc:458] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:32:54.856835 6555 mvcc.cc:205] Tried to move back new op lower bound from 7316465151405481984 to 7316465151056379904. Current Snapshot: MvccSnapshot[applied={T|T < 7316465151405481984}]
I20260809 03:32:54.856978 6548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "35d92e21b47a498796999adcfc6ba6fb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } } }
I20260809 03:32:54.857067 6548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:54.857100 6556 sys_catalog.cc:455] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "35d92e21b47a498796999adcfc6ba6fb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b478a01c8d24855948ebf23178e3c8f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40413 } } peers { permanent_uuid: "35d92e21b47a498796999adcfc6ba6fb" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 44055 } attrs { promote: false } } peers { permanent_uuid: "b505344d51174a9c9651e17bb1061ae0" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 44133 } attrs { promote: false } } }
I20260809 03:32:54.857191 6556 sys_catalog.cc:458] T 00000000000000000000000000000000 P b505344d51174a9c9651e17bb1061ae0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:32:54.862625 6549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.859069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:39613 (local address 127.0.0.1:38304)
0809 03:32:54.859190 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.859202 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:54.859257 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:54.859444 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:54.859446 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:54.859462 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:54.859516 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:54.859518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:54.860663 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:54.860664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:54.861636 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:54.861639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:54.862390 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:54.862391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:54.862453 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:54.862457 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:54.862514 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 03:32:54.862556 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260809 03:32:54.862658 6567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.859208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38304 (local address 127.27.40.129:39613)
0809 03:32:54.859326 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:54.859329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:54.859338 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:54.859368 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:54.859369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:54.859381 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:54.859451 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:54.859601 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:54.860547 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:54.861814 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:54.862303 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:54.862379 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:54.862392 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:54.862531 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 03:32:54.862575 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":61,"threads_started":1}
I20260809 03:32:54.863066 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.859346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59504 (local address 127.27.40.131:36575)
0809 03:32:54.859480 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:54.859482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:54.859491 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:54.859519 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:54.859520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:54.859531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:54.859808 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:54.859994 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:54.860688 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:54.862190 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:54.862733 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:54.862813 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:54.862827 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:54.862954 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 03:32:54.862996 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":88,"threads_started":1}
I20260809 03:32:54.863062 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.859142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:36575 (local address 127.0.0.1:59504)
0809 03:32:54.859270 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.859283 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:54.859339 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:54.859858 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:54.859859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:54.859860 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:54.859907 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:54.859908 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:54.860785 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:54.860786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:54.862098 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:54.862100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:54.862811 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:54.862812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:54.862878 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:54.862882 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:54.862940 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 03:32:54.862985 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":86,"threads_started":1}
I20260809 03:32:54.863065 6570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.859566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:44947 (local address 127.0.0.1:55126)
0809 03:32:54.859693 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.859705 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:54.859752 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:54.860168 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:54.860169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:54.860170 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:54.860205 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:54.860206 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:54.860961 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:54.860961 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:54.862054 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:54.862057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:54.862854 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:54.862855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:54.862907 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:54.862910 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:32:54.862956 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 03:32:54.862993 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":1,"thread_start_us":76,"threads_started":1}
I20260809 03:32:54.863247 6572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.859676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55126 (local address 127.27.40.130:44947)
0809 03:32:54.860010 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:54.860012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:54.860018 (+ 6us) server_negotiation.cc:407] Connection header received
0809 03:32:54.860039 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:54.860040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:54.860048 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:54.860117 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:54.860281 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:54.860871 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:54.862146 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:54.862748 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:54.862816 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:54.862827 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:54.863091 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 03:32:54.863158 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":56,"threads_started":1}
I20260809 03:32:54.863305 5923 tablet_service.cc:1467] Processing CreateTablet for tablet 94af4dc31e864cc6afff9c0e38980e6d (DEFAULT_TABLE table=test-table [id=5d0b4d8c39644e9fa6823ba31bae431e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:54.863466 5923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94af4dc31e864cc6afff9c0e38980e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:54.863509 6319 tablet_service.cc:1467] Processing CreateTablet for tablet 94af4dc31e864cc6afff9c0e38980e6d (DEFAULT_TABLE table=test-table [id=5d0b4d8c39644e9fa6823ba31bae431e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:54.863636 6319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94af4dc31e864cc6afff9c0e38980e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:54.864778 6574 tablet_bootstrap.cc:492] T 94af4dc31e864cc6afff9c0e38980e6d P 6694ec3787544426b332b9535dbe9ff3: Bootstrap starting.
I20260809 03:32:54.864915 6130 tablet_service.cc:1467] Processing CreateTablet for tablet 94af4dc31e864cc6afff9c0e38980e6d (DEFAULT_TABLE table=test-table [id=5d0b4d8c39644e9fa6823ba31bae431e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:54.865036 6130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94af4dc31e864cc6afff9c0e38980e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:54.865439 6574 tablet_bootstrap.cc:654] T 94af4dc31e864cc6afff9c0e38980e6d P 6694ec3787544426b332b9535dbe9ff3: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:54.865976 6575 tablet_bootstrap.cc:492] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3: Bootstrap starting.
I20260809 03:32:54.866348 6575 tablet_bootstrap.cc:654] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:54.866495 6577 tablet_bootstrap.cc:492] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb: Bootstrap starting.
I20260809 03:32:54.866921 6574 tablet_bootstrap.cc:492] T 94af4dc31e864cc6afff9c0e38980e6d P 6694ec3787544426b332b9535dbe9ff3: No bootstrap required, opened a new log
I20260809 03:32:54.866998 6574 ts_tablet_manager.cc:1397] T 94af4dc31e864cc6afff9c0e38980e6d P 6694ec3787544426b332b9535dbe9ff3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:32:54.867170 6575 tablet_bootstrap.cc:492] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3: No bootstrap required, opened a new log
I20260809 03:32:54.867147 6574 raft_consensus.cc:348] T 94af4dc31e864cc6afff9c0e38980e6d P 6694ec3787544426b332b9535dbe9ff3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:54.867199 6574 raft_consensus.cc:374] T 94af4dc31e864cc6afff9c0e38980e6d P 6694ec3787544426b332b9535dbe9ff3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:54.867208 6575 ts_tablet_manager.cc:1397] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:54.867213 6574 raft_consensus.cc:729] T 94af4dc31e864cc6afff9c0e38980e6d P 6694ec3787544426b332b9535dbe9ff3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6694ec3787544426b332b9535dbe9ff3, State: Initialized, Role: FOLLOWER
I20260809 03:32:54.867274 6574 consensus_queue.cc:260] T 94af4dc31e864cc6afff9c0e38980e6d P 6694ec3787544426b332b9535dbe9ff3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:54.867290 6575 raft_consensus.cc:348] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:54.867336 6575 raft_consensus.cc:374] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:54.867348 6575 raft_consensus.cc:729] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c3f42a3e8a45ee9806e74efc6ef5f3, State: Initialized, Role: FOLLOWER
I20260809 03:32:54.867405 6575 consensus_queue.cc:260] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:54.867472 6574 ts_tablet_manager.cc:1428] T 94af4dc31e864cc6afff9c0e38980e6d P 6694ec3787544426b332b9535dbe9ff3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:54.867564 6575 ts_tablet_manager.cc:1428] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:54.868053 6584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.867830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.131:54117)
0809 03:32:54.867972 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.868016 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
I20260809 03:32:54.868372 6582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.867616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.129:41275)
0809 03:32:54.868296 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.868334 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":105,"threads_started":1}
I20260809 03:32:54.868753 6586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.868552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.129:57891)
0809 03:32:54.868691 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.868722 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":91,"threads_started":1}
I20260809 03:32:54.868772 6577 tablet_bootstrap.cc:654] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:54.868847 6023 heartbeater.cc:503] Master 127.27.40.189:44055 was elected leader, sending a full tablet report...
I20260809 03:32:54.869481 6577 tablet_bootstrap.cc:492] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb: No bootstrap required, opened a new log
I20260809 03:32:54.869477 6428 heartbeater.cc:503] Master 127.27.40.189:44055 was elected leader, sending a full tablet report...
I20260809 03:32:54.869580 6577 ts_tablet_manager.cc:1397] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 03:32:54.869691 6584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.869386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.131:41057)
0809 03:32:54.869424 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.869463 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 03:32:54.869724 6577 raft_consensus.cc:348] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:54.869824 6577 raft_consensus.cc:374] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:54.869863 6577 raft_consensus.cc:729] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f20d19f3ea0c469f95ba10c2417139bb, State: Initialized, Role: FOLLOWER
I20260809 03:32:54.869938 6577 consensus_queue.cc:260] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:54.870122 6577 ts_tablet_manager.cc:1428] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:54.870643 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.870306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.130:58947)
0809 03:32:54.870538 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.870576 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":88,"threads_started":1}
I20260809 03:32:54.870733 6223 heartbeater.cc:503] Master 127.27.40.189:44055 was elected leader, sending a full tablet report...
I20260809 03:32:54.871176 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.871091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.130:59239)
0809 03:32:54.871122 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.871155 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260809 03:32:54.967988 6581 raft_consensus.cc:482] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:54.968209 6581 raft_consensus.cc:504] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:54.968436 6581 leader_election.cc:288] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6694ec3787544426b332b9535dbe9ff3 (127.27.40.129:39613), f20d19f3ea0c469f95ba10c2417139bb (127.27.40.130:44947)
I20260809 03:32:54.968731 5951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94af4dc31e864cc6afff9c0e38980e6d" candidate_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6694ec3787544426b332b9535dbe9ff3" is_pre_election: true
I20260809 03:32:54.968755 6160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94af4dc31e864cc6afff9c0e38980e6d" candidate_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f20d19f3ea0c469f95ba10c2417139bb" is_pre_election: true
I20260809 03:32:54.968855 6160 raft_consensus.cc:2454] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c3f42a3e8a45ee9806e74efc6ef5f3 in term 0.
I20260809 03:32:54.968859 5951 raft_consensus.cc:2454] T 94af4dc31e864cc6afff9c0e38980e6d P 6694ec3787544426b332b9535dbe9ff3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c3f42a3e8a45ee9806e74efc6ef5f3 in term 0.
I20260809 03:32:54.969041 6254 leader_election.cc:302] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8c3f42a3e8a45ee9806e74efc6ef5f3, f20d19f3ea0c469f95ba10c2417139bb; no voters:
I20260809 03:32:54.969156 6581 raft_consensus.cc:2781] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:54.969198 6581 raft_consensus.cc:482] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:54.969228 6581 raft_consensus.cc:3037] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:54.970024 6581 raft_consensus.cc:504] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:54.970158 6581 leader_election.cc:288] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [CANDIDATE]: Term 1 election: Requested vote from peers 6694ec3787544426b332b9535dbe9ff3 (127.27.40.129:39613), f20d19f3ea0c469f95ba10c2417139bb (127.27.40.130:44947)
I20260809 03:32:54.970328 6160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94af4dc31e864cc6afff9c0e38980e6d" candidate_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f20d19f3ea0c469f95ba10c2417139bb"
I20260809 03:32:54.970407 6160 raft_consensus.cc:3037] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:54.970430 5951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94af4dc31e864cc6afff9c0e38980e6d" candidate_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6694ec3787544426b332b9535dbe9ff3"
I20260809 03:32:54.970543 5951 raft_consensus.cc:3037] T 94af4dc31e864cc6afff9c0e38980e6d P 6694ec3787544426b332b9535dbe9ff3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:54.971172 5951 raft_consensus.cc:2454] T 94af4dc31e864cc6afff9c0e38980e6d P 6694ec3787544426b332b9535dbe9ff3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c3f42a3e8a45ee9806e74efc6ef5f3 in term 1.
I20260809 03:32:54.971216 6160 raft_consensus.cc:2454] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c3f42a3e8a45ee9806e74efc6ef5f3 in term 1.
I20260809 03:32:54.971412 6254 leader_election.cc:302] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8c3f42a3e8a45ee9806e74efc6ef5f3, f20d19f3ea0c469f95ba10c2417139bb; no voters:
I20260809 03:32:54.971513 6581 raft_consensus.cc:2781] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:54.971679 6581 raft_consensus.cc:686] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 1 LEADER]: Becoming Leader. State: Replica: b8c3f42a3e8a45ee9806e74efc6ef5f3, State: Running, Role: LEADER
I20260809 03:32:54.971725 6581 consensus_queue.cc:237] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } } peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } }
I20260809 03:32:54.972045 6584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:54.971919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.131:33549)
0809 03:32:54.971975 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:54.972017 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 03:32:54.972599 5479 catalog_manager.cc:5495] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8c3f42a3e8a45ee9806e74efc6ef5f3 (127.27.40.131). New cstate: current_term: 1 leader_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8c3f42a3e8a45ee9806e74efc6ef5f3" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 36575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 } health_report { overall_health: UNKNOWN } } }
I20260809 03:32:55.007786 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:55.005681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59512 (local address 127.27.40.131:36575)
0809 03:32:55.005755 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:55.005756 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:55.005764 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:32:55.005789 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:55.005790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:55.005800 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:55.005852 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:55.005989 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:55.006474 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:55.007212 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:55.007393 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:55.007443 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:55.007581 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:55.007608 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:55.007653 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:55.007706 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 03:32:55.007724 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 03:32:55.008836 6160 raft_consensus.cc:1261] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb [term 1 FOLLOWER]: Refusing update from remote peer b8c3f42a3e8a45ee9806e74efc6ef5f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:55.008832 5951 raft_consensus.cc:1261] T 94af4dc31e864cc6afff9c0e38980e6d P 6694ec3787544426b332b9535dbe9ff3 [term 1 FOLLOWER]: Refusing update from remote peer b8c3f42a3e8a45ee9806e74efc6ef5f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:55.009171 6581 consensus_queue.cc:1035] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6694ec3787544426b332b9535dbe9ff3" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:55.009296 6593 consensus_queue.cc:1035] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f20d19f3ea0c469f95ba10c2417139bb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:55.010243 6582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:55.010061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.129:37457)
0809 03:32:55.010170 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:55.010204 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 03:32:55.010783 6597 mvcc.cc:205] Tried to move back new op lower bound from 7316465152033890304 to 7316465151885164544. Current Snapshot: MvccSnapshot[applied={T|T < 7316465152033890304}]
I20260809 03:32:55.011032 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:55.010870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.130:49527)
0809 03:32:55.010966 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:55.010998 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 03:32:55.011379 6584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:55.011181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.131:56831)
0809 03:32:55.011320 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:55.011352 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260809 03:32:55.011420 6582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:55.011136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.129:43803)
0809 03:32:55.011341 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:55.011383 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190}
I20260809 03:32:55.012751 5479 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:55.012864 5479 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:55.013230 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:55.013117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40413 (local address 127.27.40.130:44821)
0809 03:32:55.013163 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:55.013200 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 03:32:55.014312 5479 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 35d92e21b47a498796999adcfc6ba6fb: Sending DeleteTablet for 3 replicas of tablet 94af4dc31e864cc6afff9c0e38980e6d
I20260809 03:32:55.014587 5923 tablet_service.cc:1514] Processing DeleteTablet for tablet 94af4dc31e864cc6afff9c0e38980e6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:55 UTC) from {username='oryx', principal='oryx/127.27.40.189@KRBTEST.COM'} at 127.0.0.1:38304
I20260809 03:32:55.014663 6126 tablet_service.cc:1514] Processing DeleteTablet for tablet 94af4dc31e864cc6afff9c0e38980e6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:55 UTC) from {username='oryx', principal='oryx/127.27.40.189@KRBTEST.COM'} at 127.0.0.1:55126
I20260809 03:32:55.014683 6130 tablet_service.cc:1514] Processing DeleteTablet for tablet 94af4dc31e864cc6afff9c0e38980e6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:55 UTC) from {username='oryx', principal='oryx/127.27.40.189@KRBTEST.COM'} at 127.0.0.1:55126
I20260809 03:32:55.014869 6319 tablet_service.cc:1514] Processing DeleteTablet for tablet 94af4dc31e864cc6afff9c0e38980e6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:55 UTC) from {username='oryx', principal='oryx/127.27.40.189@KRBTEST.COM'} at 127.0.0.1:59504
I20260809 03:32:55.014956 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 5694
I20260809 03:32:55.014916 6612 tablet_replica.cc:330] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb: stopping tablet replica
I20260809 03:32:55.020439 6612 raft_consensus.cc:2229] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:55.020540 6612 raft_consensus.cc:2258] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:55.021145 6613 tablet_replica.cc:330] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3: stopping tablet replica
I20260809 03:32:55.021237 6613 raft_consensus.cc:2229] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:32:55.021323 6613 raft_consensus.cc:2258] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:55.021332 6612 ts_tablet_manager.cc:1905] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:55.021685 6613 ts_tablet_manager.cc:1905] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:55.023437 6612 ts_tablet_manager.cc:1918] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:55.023525 6612 log.cc:1196] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/94af4dc31e864cc6afff9c0e38980e6d
I20260809 03:32:55.023730 6613 ts_tablet_manager.cc:1918] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:55.023795 6612 ts_tablet_manager.cc:1939] T 94af4dc31e864cc6afff9c0e38980e6d P f20d19f3ea0c469f95ba10c2417139bb: Deleting consensus metadata
I20260809 03:32:55.023841 6613 log.cc:1196] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/94af4dc31e864cc6afff9c0e38980e6d
I20260809 03:32:55.024394 6613 ts_tablet_manager.cc:1939] T 94af4dc31e864cc6afff9c0e38980e6d P b8c3f42a3e8a45ee9806e74efc6ef5f3: Deleting consensus metadata
I20260809 03:32:55.024505 5454 catalog_manager.cc:4841] TS f20d19f3ea0c469f95ba10c2417139bb (127.27.40.130:44947): tablet 94af4dc31e864cc6afff9c0e38980e6d (table test-table [id=5d0b4d8c39644e9fa6823ba31bae431e]) successfully deleted
W20260809 03:32:55.024696 5454 catalog_manager.cc:4808] TS f20d19f3ea0c469f95ba10c2417139bb (127.27.40.130:44947): delete failed for tablet 94af4dc31e864cc6afff9c0e38980e6d because the tablet was not found. No further retry: Not found: Tablet not found: 94af4dc31e864cc6afff9c0e38980e6d
I20260809 03:32:55.027081 5453 catalog_manager.cc:4841] TS b8c3f42a3e8a45ee9806e74efc6ef5f3 (127.27.40.131:36575): tablet 94af4dc31e864cc6afff9c0e38980e6d (table test-table [id=5d0b4d8c39644e9fa6823ba31bae431e]) successfully deleted
I20260809 03:32:55.033654 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 6031
I20260809 03:32:55.033697 6570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:55.033519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:39613 (local address 127.0.0.1:38320)
0809 03:32:55.033629 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:55.033663 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260809 03:32:55.036543 5457 catalog_manager.cc:4584] TS 6694ec3787544426b332b9535dbe9ff3 (127.27.40.129:39613): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94af4dc31e864cc6afff9c0e38980e6d: Network error: Client connection negotiation failed: client connection to 127.27.40.129:39613: connect: Connection refused (error 111)
I20260809 03:32:55.043663 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 6230
I20260809 03:32:55.056290 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 5277
I20260809 03:32:55.066643 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 5562
2026-08-09T03:32:55Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7215 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:32:55 dist-test-slave-gkw7 krb5kdc[6616](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:32:55 dist-test-slave-gkw7 krb5kdc[6616](info): set up 2 sockets
Aug 09 03:32:55 dist-test-slave-gkw7 krb5kdc[6616](info): commencing operation
krb5kdc: starting...
W20260809 03:32:56.360707 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:32:56 dist-test-slave-gkw7 krb5kdc[6616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:32:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:56Z Disabled control of system clock
WARNING: no policy specified for oryx/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.190.keytab.
Entry for principal oryx/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.190.keytab.
I20260809 03:32:56.492947 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:33809
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.190.keytab
--principal=oryx/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:33809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:56.578118 6637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:56.578336 6637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:56.578363 6637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:56.579556 6637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:32:56.579609 6637 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 03:32:56.579627 6637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:56.579641 6637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:56.579658 6637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:32:56.579716 6637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:32:56.581284 6637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:33809
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.190.keytab
--principal=oryx/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:33809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:56.581594 6637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:56.581863 6637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:32:56.587095 6637 server_base.cc:1034] running on GCE node
W20260809 03:32:56.587167 6644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:56.587182 6646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:56.587286 6643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:56.587546 6637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:56.587846 6637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:56.589032 6637 hybrid_clock.cc:648] HybridClock initialized: now 1786246376589009 us; error 43 us; skew 500 ppm
Aug 09 03:32:56 dist-test-slave-gkw7 krb5kdc[6616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246376, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:56.591645 6637 init.cc:377] Logged in from keytab as oryx/127.27.40.190@KRBTEST.COM (short username oryx)
I20260809 03:32:56.592134 6637 webserver.cc:460] Webserver started at http://127.27.40.190:46351/ using document root <none> and password file <none>
I20260809 03:32:56.592398 6637 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:56.592468 6637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:56.592618 6637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:56.593780 6637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "34abefd0f5d94500b9f64e7e2cbbb949"
format_stamp: "Formatted at 2026-08-09 03:32:56 on dist-test-slave-gkw7"
server_key: "995372d2f20590de4d6a2f518019eadb"
server_key_iv: "be1dbdb669679a4608b5aab36bee953e"
server_key_version: "encryptionkey@0"
I20260809 03:32:56.594157 6637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "34abefd0f5d94500b9f64e7e2cbbb949"
format_stamp: "Formatted at 2026-08-09 03:32:56 on dist-test-slave-gkw7"
server_key: "995372d2f20590de4d6a2f518019eadb"
server_key_iv: "be1dbdb669679a4608b5aab36bee953e"
server_key_version: "encryptionkey@0"
I20260809 03:32:56.595504 6637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:32:56.596225 6659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:56.596400 6637 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:56.596495 6637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "34abefd0f5d94500b9f64e7e2cbbb949"
format_stamp: "Formatted at 2026-08-09 03:32:56 on dist-test-slave-gkw7"
server_key: "995372d2f20590de4d6a2f518019eadb"
server_key_iv: "be1dbdb669679a4608b5aab36bee953e"
server_key_version: "encryptionkey@0"
I20260809 03:32:56.596578 6637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:56.627302 6637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:56.629362 6637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:56.629585 6637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:56.634725 6637 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:33809
I20260809 03:32:56.634768 6743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:33809 every 8 connection(s)
I20260809 03:32:56.635106 6637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:32:56.635859 6744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:56.638502 6744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949: Bootstrap starting.
I20260809 03:32:56.639148 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 6637
I20260809 03:32:56.639225 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:32:56.639213 6744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:56.639425 27810 external_mini_cluster.cc:1428] Setting key b37958f8d82fbaf46740057baa33c0f1
I20260809 03:32:56.639609 6744 log.cc:824] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:56.640515 6744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949: No bootstrap required, opened a new log
I20260809 03:32:56.642753 6744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34abefd0f5d94500b9f64e7e2cbbb949" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 33809 } }
I20260809 03:32:56.642946 6744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:56.643005 6744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34abefd0f5d94500b9f64e7e2cbbb949, State: Initialized, Role: FOLLOWER
I20260809 03:32:56.643181 6744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34abefd0f5d94500b9f64e7e2cbbb949" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 33809 } }
I20260809 03:32:56.643288 6744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:32:56.643342 6744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:32:56.643399 6744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:56.644222 6744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34abefd0f5d94500b9f64e7e2cbbb949" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 33809 } }
I20260809 03:32:56.644361 6744 leader_election.cc:302] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34abefd0f5d94500b9f64e7e2cbbb949; no voters:
I20260809 03:32:56.644598 6744 leader_election.cc:288] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:32:56.644629 6752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:56.644827 6752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [term 1 LEADER]: Becoming Leader. State: Replica: 34abefd0f5d94500b9f64e7e2cbbb949, State: Running, Role: LEADER
I20260809 03:32:56.644987 6744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:32:56.644992 6752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34abefd0f5d94500b9f64e7e2cbbb949" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 33809 } }
I20260809 03:32:56.645574 6753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34abefd0f5d94500b9f64e7e2cbbb949. Latest consensus state: current_term: 1 leader_uuid: "34abefd0f5d94500b9f64e7e2cbbb949" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34abefd0f5d94500b9f64e7e2cbbb949" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 33809 } } }
I20260809 03:32:56.645679 6753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [sys.catalog]: This master's current role is: LEADER
I20260809 03:32:56.645931 6763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:32:56.646576 6763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:32:56.646526 6756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34abefd0f5d94500b9f64e7e2cbbb949" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34abefd0f5d94500b9f64e7e2cbbb949" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 33809 } } }
I20260809 03:32:56.646770 6756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949 [sys.catalog]: This master's current role is: LEADER
Aug 09 03:32:56 dist-test-slave-gkw7 krb5kdc[6616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM
I20260809 03:32:56.649988 6763 catalog_manager.cc:1347] Generated new cluster ID: 0581e009129d42349b1a2c90d243f749
I20260809 03:32:56.650072 6763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:32:56.651183 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:56.640079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49378 (local address 127.27.40.190:33809)
0809 03:32:56.640296 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:56.640305 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:56.640340 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:32:56.640802 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:56.640808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:56.640857 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:56.641012 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:56.641694 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:56.642600 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:56.643482 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:56.644214 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:56.644363 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:56.647638 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:56.647642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:56.647643 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:56.647654 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:56.649458 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:56.649861 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:56.649864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:56.649865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:56.649973 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:56.650212 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:56.650214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:56.650214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:56.650557 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:56.650717 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:56.650836 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 03:32:56.650978 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":102,"threads_started":1}
I20260809 03:32:56.674017 6763 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:32:56.674787 6763 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:32:56.683084 6763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949: Generated new TSK 0
I20260809 03:32:56.683274 6763 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.129.keytab.
Entry for principal oryx/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.129.keytab.
I20260809 03:32:56.714771 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:33809
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.129.keytab
--principal=oryx/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:56.798918 6782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:56.799144 6782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:56.799185 6782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:56.800352 6782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:56.800417 6782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:56.800453 6782 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:32:56.800508 6782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:32:56.802070 6782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.129.keytab
--principal=oryx/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.40.190:33809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:56.802325 6782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:56.802554 6782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:56.807512 6791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:56.807561 6792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:56.807570 6795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:56.807730 6782 server_base.cc:1034] running on GCE node
I20260809 03:32:56.807938 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:56.808180 6782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:56.809360 6782 hybrid_clock.cc:648] HybridClock initialized: now 1786246376809340 us; error 30 us; skew 500 ppm
Aug 09 03:32:56 dist-test-slave-gkw7 krb5kdc[6616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246376, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:56.811945 6782 init.cc:377] Logged in from keytab as oryx/127.27.40.129@KRBTEST.COM (short username oryx)
I20260809 03:32:56.812372 6782 webserver.cc:460] Webserver started at http://127.27.40.129:45059/ using document root <none> and password file <none>
I20260809 03:32:56.812595 6782 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:56.812668 6782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:56.812820 6782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:56.813944 6782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "1e4a589170ce4831801b40e9a9bcf454"
format_stamp: "Formatted at 2026-08-09 03:32:56 on dist-test-slave-gkw7"
server_key: "13be2eed59407f1bf5e11c311e70d9bb"
server_key_iv: "fac0df54a55374cb980c6cc204cc7ea5"
server_key_version: "encryptionkey@0"
I20260809 03:32:56.814303 6782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "1e4a589170ce4831801b40e9a9bcf454"
format_stamp: "Formatted at 2026-08-09 03:32:56 on dist-test-slave-gkw7"
server_key: "13be2eed59407f1bf5e11c311e70d9bb"
server_key_iv: "fac0df54a55374cb980c6cc204cc7ea5"
server_key_version: "encryptionkey@0"
I20260809 03:32:56.815649 6782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:56.816347 6806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:56.816517 6782 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:56.816634 6782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "1e4a589170ce4831801b40e9a9bcf454"
format_stamp: "Formatted at 2026-08-09 03:32:56 on dist-test-slave-gkw7"
server_key: "13be2eed59407f1bf5e11c311e70d9bb"
server_key_iv: "fac0df54a55374cb980c6cc204cc7ea5"
server_key_version: "encryptionkey@0"
I20260809 03:32:56.816707 6782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:56.844419 6782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:56.846987 6782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:56.847112 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:56.847613 6782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:56.847683 6782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:56.847735 6782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:56.847759 6782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:56.855020 6782 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:42023
Aug 09 03:32:56 dist-test-slave-gkw7 krb5kdc[6616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246376, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM
I20260809 03:32:56.855098 6978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:42023 every 8 connection(s)
I20260809 03:32:56.855430 6782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:32:56.858347 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:56.848211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:60463 (local address 127.27.40.190:33809)
0809 03:32:56.848257 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:56.848262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:56.848513 (+ 251us) server_negotiation.cc:407] Connection header received
0809 03:32:56.849078 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:56.849080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:56.849097 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:56.849181 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:56.850229 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:56.851082 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:56.852811 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:56.853051 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:56.853110 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:56.855619 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:56.855620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:56.855621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:56.855629 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:56.856885 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:56.857288 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:56.857289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:56.857289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:56.857355 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:56.857670 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:56.857671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:56.857672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:56.857793 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:56.857873 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:56.858161 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 03:32:56.858255 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 03:32:56.858512 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:56.848198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:33809 (local address 127.27.40.129:60463)
0809 03:32:56.848384 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:56.848420 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:56.848964 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:56.849301 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:56.849306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:56.849774 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:56.850100 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:56.850107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:56.851187 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:56.851188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:56.852698 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:56.852701 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:56.853136 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:56.853137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:56.853307 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:56.853928 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:56.853936 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:56.855503 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:56.856972 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:56.856980 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:56.856982 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:56.857208 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:56.857428 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:56.857428 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:56.857429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:56.857597 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:56.857864 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:56.857871 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:56.858013 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 03:32:56.858115 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 03:32:56.858280 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":97,"threads_started":1}
I20260809 03:32:56.859030 6979 heartbeater.cc:347] Connected to a master server at 127.27.40.190:33809
I20260809 03:32:56.859115 6979 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:56.859300 6979 heartbeater.cc:511] Master 127.27.40.190:33809 requested a full tablet report, sending...
I20260809 03:32:56.859869 6681 ts_manager.cc:194] Registered new tserver with Master: 1e4a589170ce4831801b40e9a9bcf454 (127.27.40.129:42023)
I20260809 03:32:56.860718 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 6782
I20260809 03:32:56.860827 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:32:56.860848 6681 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:60463
I20260809 03:32:56.861061 27810 external_mini_cluster.cc:1428] Setting key 399404c7736a5531dfcb361b345af391
I20260809 03:32:56.865942 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:56.860070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:57841 (local address 127.27.40.190:33809)
0809 03:32:56.860109 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:56.860110 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:56.860355 (+ 245us) server_negotiation.cc:407] Connection header received
0809 03:32:56.860379 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:56.860380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:56.860391 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:56.860455 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:56.860739 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:56.861437 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:56.862242 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:56.862435 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:56.862511 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:56.863386 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:56.863388 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:56.863388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:56.863395 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:56.864775 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:56.865129 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:56.865130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:56.865131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:56.865186 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:56.865399 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:56.865400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:56.865401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:56.865514 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:56.865614 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:56.865770 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 03:32:56.865846 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:32:56.865954 6985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:56.860142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:33809 (local address 127.27.40.129:57841)
0809 03:32:56.860281 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:56.860291 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:56.860345 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:56.860458 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:56.860458 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:56.860610 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:56.860659 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:56.860660 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:56.861550 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:56.861551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:56.862109 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:56.862111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:56.862521 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:56.862522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:56.862598 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:56.863014 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:56.863017 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:56.863297 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:56.864879 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:56.864880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:56.864881 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:56.865055 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:56.865266 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:56.865267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:56.865267 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:56.865333 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:56.865605 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:56.865606 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:56.865655 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 03:32:56.865744 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 03:32:56.865849 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for oryx/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.130.keytab.
Entry for principal oryx/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.130.keytab.
I20260809 03:32:56.889192 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:33809
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.130.keytab
--principal=oryx/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:56.974480 6988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:56.974685 6988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:56.974711 6988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:56.975804 6988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:56.975844 6988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:56.975865 6988 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:32:56.975884 6988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:32:56.977355 6988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.130.keytab
--principal=oryx/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.40.190:33809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:56.977607 6988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:56.977845 6988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:56.982980 6995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:56.982980 6994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:56.983397 6988 server_base.cc:1034] running on GCE node
W20260809 03:32:56.983050 6997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:56.983669 6988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:56.983925 6988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:56.985066 6988 hybrid_clock.cc:648] HybridClock initialized: now 1786246376985048 us; error 44 us; skew 500 ppm
Aug 09 03:32:56 dist-test-slave-gkw7 krb5kdc[6616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246376, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:56.987702 6988 init.cc:377] Logged in from keytab as oryx/127.27.40.130@KRBTEST.COM (short username oryx)
I20260809 03:32:56.988094 6988 webserver.cc:460] Webserver started at http://127.27.40.130:34643/ using document root <none> and password file <none>
I20260809 03:32:56.988278 6988 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:56.988319 6988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:56.988409 6988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:56.989393 6988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "1e4a9769605c4902b987b27d1d20c995"
format_stamp: "Formatted at 2026-08-09 03:32:56 on dist-test-slave-gkw7"
server_key: "01b7aa67b18723d2f880b48d6349e1a0"
server_key_iv: "848ab21a7980f58d499e056a50efc6e2"
server_key_version: "encryptionkey@0"
I20260809 03:32:56.989851 6988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "1e4a9769605c4902b987b27d1d20c995"
format_stamp: "Formatted at 2026-08-09 03:32:56 on dist-test-slave-gkw7"
server_key: "01b7aa67b18723d2f880b48d6349e1a0"
server_key_iv: "848ab21a7980f58d499e056a50efc6e2"
server_key_version: "encryptionkey@0"
I20260809 03:32:56.991134 6988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:56.991816 7004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:56.992012 6988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:56.992101 6988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "1e4a9769605c4902b987b27d1d20c995"
format_stamp: "Formatted at 2026-08-09 03:32:56 on dist-test-slave-gkw7"
server_key: "01b7aa67b18723d2f880b48d6349e1a0"
server_key_iv: "848ab21a7980f58d499e056a50efc6e2"
server_key_version: "encryptionkey@0"
I20260809 03:32:56.992156 6988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:57.002453 6988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:57.004913 6988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:57.005031 6988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:57.005496 6988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:57.005581 6988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:57.005625 6988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:57.005653 6988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:57.012705 6988 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:32837
Aug 09 03:32:57 dist-test-slave-gkw7 krb5kdc[6616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246376, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM
I20260809 03:32:57.012763 7167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:32837 every 8 connection(s)
I20260809 03:32:57.013072 6988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:32:57.014901 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 6988
I20260809 03:32:57.014992 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:32:57.015220 27810 external_mini_cluster.cc:1428] Setting key 2b9d804d9bad09f8d2aa9ea74963cb8a
I20260809 03:32:57.016188 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.005999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:38619 (local address 127.27.40.190:33809)
0809 03:32:57.006039 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.006041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.006276 (+ 235us) server_negotiation.cc:407] Connection header received
0809 03:32:57.006805 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.006807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.006821 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.006872 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:57.007676 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.008489 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.010222 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.010547 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.010642 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.013234 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:57.013236 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:57.013236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:57.013244 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:57.014669 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:57.015112 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:57.015114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:57.015114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:57.015181 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:57.015458 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:57.015459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:57.015460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:57.015601 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:57.015698 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:57.016006 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 03:32:57.016098 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 03:32:57.016387 7015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.005990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:33809 (local address 127.27.40.130:38619)
0809 03:32:57.006154 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:57.006188 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:57.006691 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:57.006937 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:57.006941 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:57.007298 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:57.007579 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.007585 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.008610 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.008611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.010123 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.010127 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.010644 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.010645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.010795 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.011338 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:57.011349 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:57.013150 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:57.014767 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:57.014773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:57.014776 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:57.015049 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:57.015252 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:57.015253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:57.015254 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:57.015380 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:57.015687 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:57.015694 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:57.015853 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 03:32:57.015982 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 03:32:57.016149 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260809 03:32:57.016875 7169 heartbeater.cc:347] Connected to a master server at 127.27.40.190:33809
I20260809 03:32:57.016973 7169 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:57.017187 7169 heartbeater.cc:511] Master 127.27.40.190:33809 requested a full tablet report, sending...
I20260809 03:32:57.017658 6682 ts_manager.cc:194] Registered new tserver with Master: 1e4a9769605c4902b987b27d1d20c995 (127.27.40.130:32837)
I20260809 03:32:57.018196 6682 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:38619
I20260809 03:32:57.024821 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.018613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:59153 (local address 127.27.40.190:33809)
0809 03:32:57.018665 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.018667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.018806 (+ 139us) server_negotiation.cc:407] Connection header received
0809 03:32:57.018838 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.018839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.018851 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.018913 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:57.019255 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.019844 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.020968 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.021151 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.021204 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.022321 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:57.022322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:57.022322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:57.022328 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:57.023664 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:57.024071 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:57.024072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:57.024072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:57.024120 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:57.024343 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:57.024344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:57.024344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:57.024472 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:57.024523 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:57.024706 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 03:32:57.024761 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 03:32:57.024919 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.018569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:33809 (local address 127.27.40.130:59153)
0809 03:32:57.018704 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:57.018715 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:57.018796 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:57.018917 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:57.018918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:57.019111 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:57.019171 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.019172 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.019938 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.019940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.020887 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.020890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.021228 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.021229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.021305 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.021775 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:57.021795 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:57.022234 (+ 439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:57.023750 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:57.023751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:57.023752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:57.023995 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:57.024203 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:57.024204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:57.024205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:57.024276 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:57.024538 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:57.024539 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:57.024591 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 03:32:57.024691 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 03:32:57.024801 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for oryx/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.131.keytab.
Entry for principal oryx/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.131.keytab.
I20260809 03:32:57.044819 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:33809
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.131.keytab
--principal=oryx/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:32:57.130149 7183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:32:57.130360 7183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:32:57.130388 7183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:32:57.131608 7183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:32:57.131659 7183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:32:57.131681 7183 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:32:57.131739 7183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:32:57.133266 7183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.131.keytab
--principal=oryx/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.40.190:33809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:32:57.133520 7183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:32:57.133803 7183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:32:57.138901 7194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:57.139019 7196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:32:57.138906 7193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:32:57.139129 7183 server_base.cc:1034] running on GCE node
I20260809 03:32:57.139447 7183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:32:57.139668 7183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:32:57.140811 7183 hybrid_clock.cc:648] HybridClock initialized: now 1786246377140800 us; error 34 us; skew 500 ppm
Aug 09 03:32:57 dist-test-slave-gkw7 krb5kdc[6616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246377, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:32:57.143404 7183 init.cc:377] Logged in from keytab as oryx/127.27.40.131@KRBTEST.COM (short username oryx)
I20260809 03:32:57.143818 7183 webserver.cc:460] Webserver started at http://127.27.40.131:45903/ using document root <none> and password file <none>
I20260809 03:32:57.144050 7183 fs_manager.cc:359] Metadata directory not provided
I20260809 03:32:57.144125 7183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:32:57.144275 7183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:32:57.145637 7183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "25a8cee434324accb84886ef0e91cd27"
format_stamp: "Formatted at 2026-08-09 03:32:57 on dist-test-slave-gkw7"
server_key: "216f7070c21601e270751fa3850fd169"
server_key_iv: "fdd701a8edc46a994766d2cf2a3cc061"
server_key_version: "encryptionkey@0"
I20260809 03:32:57.146131 7183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "25a8cee434324accb84886ef0e91cd27"
format_stamp: "Formatted at 2026-08-09 03:32:57 on dist-test-slave-gkw7"
server_key: "216f7070c21601e270751fa3850fd169"
server_key_iv: "fdd701a8edc46a994766d2cf2a3cc061"
server_key_version: "encryptionkey@0"
I20260809 03:32:57.147684 7183 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 03:32:57.148473 7205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:57.148659 7183 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:32:57.148770 7183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "25a8cee434324accb84886ef0e91cd27"
format_stamp: "Formatted at 2026-08-09 03:32:57 on dist-test-slave-gkw7"
server_key: "216f7070c21601e270751fa3850fd169"
server_key_iv: "fdd701a8edc46a994766d2cf2a3cc061"
server_key_version: "encryptionkey@0"
I20260809 03:32:57.148847 7183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:32:57.193907 7183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:32:57.196173 7183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:32:57.196353 7183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:32:57.196940 7183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:32:57.197008 7183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:57.197055 7183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:32:57.197086 7183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:32:57 dist-test-slave-gkw7 krb5kdc[6616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246377, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM
I20260809 03:32:57.207517 7183 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:42861
I20260809 03:32:57.207726 7380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:42861 every 8 connection(s)
I20260809 03:32:57.208039 7183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:32:57.208709 7219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.197417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:33809 (local address 127.27.40.131:47419)
0809 03:32:57.197643 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:57.197705 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:57.198518 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:57.198827 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:57.198832 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:57.199294 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:57.199659 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.199667 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.200454 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.200455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.202224 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.202227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.202667 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.202668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.202798 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.203302 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:57.203316 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:57.205267 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:57.206986 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:57.206993 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:57.206995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:57.207299 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:57.207546 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:57.207547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:57.207547 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:57.207671 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:57.208019 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:57.208025 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:57.208192 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 03:32:57.208320 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 03:32:57.208484 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":110,"threads_started":1}
I20260809 03:32:57.209010 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.197434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:47419 (local address 127.27.40.190:33809)
0809 03:32:57.197508 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.197511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.197813 (+ 302us) server_negotiation.cc:407] Connection header received
0809 03:32:57.198652 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.198653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.198685 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.198741 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:57.199761 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.200361 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.202321 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.202546 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.202634 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.205372 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:57.205374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:57.205374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:57.205382 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:57.206878 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:57.207398 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:57.207399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:57.207400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:57.207468 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:57.207756 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:57.207758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:57.207758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:57.207939 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:57.208008 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:57.208831 (+ 823us) server_negotiation.cc:299] Negotiation successful
0809 03:32:57.208923 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:32:57.209637 7381 heartbeater.cc:347] Connected to a master server at 127.27.40.190:33809
I20260809 03:32:57.209757 7381 heartbeater.cc:464] Registering TS with master...
I20260809 03:32:57.209946 7381 heartbeater.cc:511] Master 127.27.40.190:33809 requested a full tablet report, sending...
I20260809 03:32:57.210418 6681 ts_manager.cc:194] Registered new tserver with Master: 25a8cee434324accb84886ef0e91cd27 (127.27.40.131:42861)
I20260809 03:32:57.210994 6681 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:47419
I20260809 03:32:57.211035 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 7183
I20260809 03:32:57.211136 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:32:57.211386 27810 external_mini_cluster.cc:1428] Setting key 0b455a5ae83c2bc85a5f3589af25fb43
I20260809 03:32:57.212265 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:32:57.217497 7394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.210658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:33809 (local address 127.27.40.131:34927)
0809 03:32:57.210916 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:57.210929 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:57.210996 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:57.211187 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:57.211188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:57.211405 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:57.211485 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.211487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.212420 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.212421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.212959 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.212961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.213335 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.213336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.213458 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.214030 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:57.214035 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:57.214422 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:57.216442 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:57.216443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:57.216444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:57.216638 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:57.216837 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:57.216837 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:57.216838 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:57.216935 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:57.217256 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:57.217256 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:57.217298 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 03:32:57.217354 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 03:32:57.217432 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":129,"threads_started":1}
I20260809 03:32:57.217651 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.210716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:34927 (local address 127.27.40.190:33809)
0809 03:32:57.210772 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.210774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.211003 (+ 229us) server_negotiation.cc:407] Connection header received
0809 03:32:57.211053 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.211054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.211065 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.211139 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:57.211553 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.212161 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.213046 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.213249 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.213332 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.214579 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:57.214580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:57.214581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:57.214588 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:57.216360 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:57.216712 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:57.216713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:57.216714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:57.216773 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:57.217011 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:57.217012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:57.217012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:57.217192 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:57.217253 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:57.217403 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 03:32:57.217483 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
Aug 09 03:32:57 dist-test-slave-gkw7 krb5kdc[6616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246377, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 03:32:57 dist-test-slave-gkw7 krb5kdc[6616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:32:57 dist-test-slave-gkw7 krb5kdc[6616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 03:32:57.238907 6751 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:32:57.233008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49394 (local address 127.27.40.190:33809)
0809 03:32:57.233105 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.233107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.233129 (+ 22us) server_negotiation.cc:407] Connection header received
0809 03:32:57.233163 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.233164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.233175 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.233238 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:57.233986 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.234494 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.235292 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.235486 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.235571 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.238823 (+ 3252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49394: BlockingRecv error: recv got EOF from 127.0.0.1:49394 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 03:32:57 dist-test-slave-gkw7 krb5kdc[6616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246377, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM
I20260809 03:32:57.246392 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.239670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49404 (local address 127.27.40.190:33809)
0809 03:32:57.239733 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.239735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.239802 (+ 67us) server_negotiation.cc:407] Connection header received
0809 03:32:57.239890 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.239891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.239901 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.239975 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:57.240502 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.240965 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.241722 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.241921 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.241997 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.244049 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:57.244050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:57.244050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:57.244056 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:57.245337 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:57.245732 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:57.245733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:57.245734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:57.245799 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:57.245965 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:57.245966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:57.245966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:57.246055 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:57.246153 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:57.246274 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 03:32:57.246332 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 03:32:57.247901 6681 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:32:57.249574 6681 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:57.256934 7416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.251979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:42023 (local address 127.0.0.1:46294)
0809 03:32:57.252528 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:57.252539 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:57.252592 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:57.252948 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:57.252950 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:57.252953 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:57.253023 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.253026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.254751 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.254752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.255861 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.255864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.256565 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.256566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.256662 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.256677 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 03:32:57.256771 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 03:32:57.256832 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":53,"thread_start_us":289,"threads_started":1}
I20260809 03:32:57.256934 7415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.251936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:32837 (local address 127.0.0.1:43316)
0809 03:32:57.252217 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:57.252259 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:57.252331 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:57.252866 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:57.252869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:57.252875 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:57.252947 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.252949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.254297 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.254298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.255566 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.255569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.256725 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.256726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.256799 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.256802 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:32:57.256844 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 03:32:57.256874 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":225,"threads_started":1}
I20260809 03:32:57.257187 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.252251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43316 (local address 127.27.40.130:32837)
0809 03:32:57.252524 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.252527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.252537 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:32:57.252725 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.252728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.252747 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.253026 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:57.253111 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.254204 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.255668 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.256555 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.256704 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.256861 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:57.256893 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 03:32:57.256934 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":131,"threads_started":1}
I20260809 03:32:57.257452 7417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.251936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:42861 (local address 127.0.0.1:33800)
0809 03:32:57.252480 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:57.252501 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:57.252559 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:57.253111 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:57.253111 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:57.253113 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:57.253150 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.253151 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.254743 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.254744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.255585 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.255588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.256906 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.256907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.256968 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.256972 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:57.257314 (+ 342us) client_negotiation.cc:240] Negotiation successful
0809 03:32:57.257385 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":43,"thread_start_us":51,"threads_started":1}
I20260809 03:32:57.257845 7420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.252500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33800 (local address 127.27.40.131:42861)
0809 03:32:57.252904 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.252909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.252919 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:32:57.252988 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.252990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.253007 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.253089 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:57.253270 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.254641 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.255711 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.256639 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.256704 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.256865 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:57.257695 (+ 830us) server_negotiation.cc:299] Negotiation successful
0809 03:32:57.257728 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":86,"threads_started":1}
I20260809 03:32:57.257831 7418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.252153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46294 (local address 127.27.40.129:42023)
0809 03:32:57.252413 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.252418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.252697 (+ 279us) server_negotiation.cc:407] Connection header received
0809 03:32:57.252793 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.252795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.252829 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.253104 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:57.253168 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.254653 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.255947 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.256462 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.256551 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.256631 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:57.257405 (+ 774us) server_negotiation.cc:299] Negotiation successful
0809 03:32:57.257453 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":128,"threads_started":1}
I20260809 03:32:57.258065 7071 tablet_service.cc:1467] Processing CreateTablet for tablet a6645ce3ce81415a8d0d3d393230d755 (DEFAULT_TABLE table=test-table [id=e13822dd0833417c8d24978376bd383e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:57.258575 7071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6645ce3ce81415a8d0d3d393230d755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:57.258754 7261 tablet_service.cc:1467] Processing CreateTablet for tablet a6645ce3ce81415a8d0d3d393230d755 (DEFAULT_TABLE table=test-table [id=e13822dd0833417c8d24978376bd383e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:57.259166 7261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6645ce3ce81415a8d0d3d393230d755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:57.262301 6888 tablet_service.cc:1467] Processing CreateTablet for tablet a6645ce3ce81415a8d0d3d393230d755 (DEFAULT_TABLE table=test-table [id=e13822dd0833417c8d24978376bd383e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:57.262701 6888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6645ce3ce81415a8d0d3d393230d755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:57.264786 7422 tablet_bootstrap.cc:492] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27: Bootstrap starting.
I20260809 03:32:57.265892 7421 tablet_bootstrap.cc:492] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995: Bootstrap starting.
I20260809 03:32:57.266479 7422 tablet_bootstrap.cc:654] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:57.267076 7421 tablet_bootstrap.cc:654] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:57.267859 7421 log.cc:824] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:57.267839 7422 log.cc:824] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:57.268729 7421 tablet_bootstrap.cc:492] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995: No bootstrap required, opened a new log
I20260809 03:32:57.268818 7421 ts_tablet_manager.cc:1397] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:32:57.269331 7427 tablet_bootstrap.cc:492] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454: Bootstrap starting.
I20260809 03:32:57.270318 7422 tablet_bootstrap.cc:492] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27: No bootstrap required, opened a new log
I20260809 03:32:57.270519 7422 ts_tablet_manager.cc:1397] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 03:32:57.270998 7427 tablet_bootstrap.cc:654] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:57.271864 7427 log.cc:824] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454: Log is configured to *not* fsync() on all Append() calls
I20260809 03:32:57.273851 7421 raft_consensus.cc:348] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } }
I20260809 03:32:57.274116 7421 raft_consensus.cc:374] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:57.274197 7421 raft_consensus.cc:729] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4a9769605c4902b987b27d1d20c995, State: Initialized, Role: FOLLOWER
I20260809 03:32:57.274379 7421 consensus_queue.cc:260] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } }
I20260809 03:32:57.274825 7421 ts_tablet_manager.cc:1428] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260809 03:32:57.275025 7169 heartbeater.cc:503] Master 127.27.40.190:33809 was elected leader, sending a full tablet report...
I20260809 03:32:57.275027 7427 tablet_bootstrap.cc:492] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454: No bootstrap required, opened a new log
I20260809 03:32:57.275279 7427 ts_tablet_manager.cc:1397] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:32:57.275542 7422 raft_consensus.cc:348] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } }
I20260809 03:32:57.275717 7422 raft_consensus.cc:374] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:57.275759 7422 raft_consensus.cc:729] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a8cee434324accb84886ef0e91cd27, State: Initialized, Role: FOLLOWER
I20260809 03:32:57.275866 7422 consensus_queue.cc:260] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } }
I20260809 03:32:57.276268 7422 ts_tablet_manager.cc:1428] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260809 03:32:57.276432 7381 heartbeater.cc:503] Master 127.27.40.190:33809 was elected leader, sending a full tablet report...
I20260809 03:32:57.280581 7427 raft_consensus.cc:348] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } }
I20260809 03:32:57.280762 7427 raft_consensus.cc:374] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:57.280808 7427 raft_consensus.cc:729] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4a589170ce4831801b40e9a9bcf454, State: Initialized, Role: FOLLOWER
I20260809 03:32:57.280933 7427 consensus_queue.cc:260] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } }
I20260809 03:32:57.281339 7427 ts_tablet_manager.cc:1428] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260809 03:32:57.281442 6979 heartbeater.cc:503] Master 127.27.40.190:33809 was elected leader, sending a full tablet report...
W20260809 03:32:57.356338 6980 tablet.cc:2369] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:32:57.458813 7383 tablet.cc:2369] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:32:57.514010 7171 tablet.cc:2369] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:32:57.573511 7430 raft_consensus.cc:482] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:57.573737 7430 raft_consensus.cc:504] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } }
I20260809 03:32:57.574194 7430 leader_election.cc:288] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25a8cee434324accb84886ef0e91cd27 (127.27.40.131:42861), 1e4a589170ce4831801b40e9a9bcf454 (127.27.40.129:42023)
I20260809 03:32:57.578811 7418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.574455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:46109 (local address 127.27.40.129:42023)
0809 03:32:57.574520 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.574522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.574638 (+ 116us) server_negotiation.cc:407] Connection header received
0809 03:32:57.574728 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.574730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.574742 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.574824 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:57.575092 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.575918 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.577758 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.578320 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.578431 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.578442 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:57.578680 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 03:32:57.578720 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 03:32:57.578858 7436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.574391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:42023 (local address 127.27.40.130:46109)
0809 03:32:57.574546 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:57.574559 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:57.574656 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:57.574852 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:57.574853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:57.574855 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:57.574983 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.574985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.576022 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.576023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.577652 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.577655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.578443 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.578444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.578552 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.578557 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:32:57.578665 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 03:32:57.578732 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":50,"thread_start_us":80,"threads_started":1}
I20260809 03:32:57.578934 7420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.574746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:45063 (local address 127.27.40.131:42861)
0809 03:32:57.574829 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.574832 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.574841 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:57.574868 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.574869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.574877 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.574929 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:57.575189 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.575918 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.577758 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.578450 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.578516 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.578532 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:57.578812 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 03:32:57.578860 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:32:57.579247 7438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.574420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:42861 (local address 127.27.40.130:45063)
0809 03:32:57.574594 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:57.574606 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:57.574667 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:57.574954 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:57.574955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:57.574956 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:57.575098 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.575100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.576022 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.576023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.577652 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.577655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.578528 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.578529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.578727 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.578731 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:57.578790 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 03:32:57.578826 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":55,"thread_start_us":61,"threads_started":1}
I20260809 03:32:57.579489 6912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6645ce3ce81415a8d0d3d393230d755" candidate_uuid: "1e4a9769605c4902b987b27d1d20c995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4a589170ce4831801b40e9a9bcf454" is_pre_election: true
I20260809 03:32:57.579667 6912 raft_consensus.cc:2454] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e4a9769605c4902b987b27d1d20c995 in term 0.
I20260809 03:32:57.579799 7317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6645ce3ce81415a8d0d3d393230d755" candidate_uuid: "1e4a9769605c4902b987b27d1d20c995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25a8cee434324accb84886ef0e91cd27" is_pre_election: true
I20260809 03:32:57.579942 7007 leader_election.cc:302] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e4a589170ce4831801b40e9a9bcf454, 1e4a9769605c4902b987b27d1d20c995; no voters:
I20260809 03:32:57.579984 7317 raft_consensus.cc:2454] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e4a9769605c4902b987b27d1d20c995 in term 0.
I20260809 03:32:57.580114 7430 raft_consensus.cc:2781] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:57.580178 7430 raft_consensus.cc:482] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:57.580211 7430 raft_consensus.cc:3037] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:57.581314 7430 raft_consensus.cc:504] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } }
I20260809 03:32:57.581494 7430 leader_election.cc:288] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [CANDIDATE]: Term 1 election: Requested vote from peers 25a8cee434324accb84886ef0e91cd27 (127.27.40.131:42861), 1e4a589170ce4831801b40e9a9bcf454 (127.27.40.129:42023)
I20260809 03:32:57.581676 6912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6645ce3ce81415a8d0d3d393230d755" candidate_uuid: "1e4a9769605c4902b987b27d1d20c995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4a589170ce4831801b40e9a9bcf454"
I20260809 03:32:57.581719 7317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6645ce3ce81415a8d0d3d393230d755" candidate_uuid: "1e4a9769605c4902b987b27d1d20c995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25a8cee434324accb84886ef0e91cd27"
I20260809 03:32:57.581768 6912 raft_consensus.cc:3037] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:57.581825 7317 raft_consensus.cc:3037] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:57.582677 6912 raft_consensus.cc:2454] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e4a9769605c4902b987b27d1d20c995 in term 1.
I20260809 03:32:57.582880 7007 leader_election.cc:302] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e4a589170ce4831801b40e9a9bcf454, 1e4a9769605c4902b987b27d1d20c995; no voters:
I20260809 03:32:57.582974 7430 raft_consensus.cc:2781] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:57.583146 7430 raft_consensus.cc:686] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [term 1 LEADER]: Becoming Leader. State: Replica: 1e4a9769605c4902b987b27d1d20c995, State: Running, Role: LEADER
I20260809 03:32:57.583249 7430 consensus_queue.cc:237] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } }
I20260809 03:32:57.583477 7317 raft_consensus.cc:2454] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e4a9769605c4902b987b27d1d20c995 in term 1.
I20260809 03:32:57.584128 6682 catalog_manager.cc:5495] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e4a9769605c4902b987b27d1d20c995 (127.27.40.130). New cstate: current_term: 1 leader_uuid: "1e4a9769605c4902b987b27d1d20c995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } health_report { overall_health: UNKNOWN } } }
I20260809 03:32:57.611856 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.608979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43332 (local address 127.27.40.130:32837)
0809 03:32:57.609034 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.609036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.609047 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:32:57.609149 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.609150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.609175 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.609272 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:57.609460 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.610053 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.610946 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.611176 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.611271 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.611473 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:57.611599 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:57.611701 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:57.611733 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 03:32:57.611760 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 03:32:57.614217 6912 raft_consensus.cc:1261] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454 [term 1 FOLLOWER]: Refusing update from remote peer 1e4a9769605c4902b987b27d1d20c995: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:57.614225 7317 raft_consensus.cc:1261] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Refusing update from remote peer 1e4a9769605c4902b987b27d1d20c995: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:57.614650 7440 consensus_queue.cc:1035] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:57.614786 7430 consensus_queue.cc:1035] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:57.620775 6681 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:57.620900 6681 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:57.621804 6681 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949: Sending DeleteTablet for 3 replicas of tablet a6645ce3ce81415a8d0d3d393230d755
I20260809 03:32:57.622217 7261 tablet_service.cc:1514] Processing DeleteTablet for tablet a6645ce3ce81415a8d0d3d393230d755 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:57 UTC) from {username='oryx', principal='oryx/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:33800
I20260809 03:32:57.622262 6888 tablet_service.cc:1514] Processing DeleteTablet for tablet a6645ce3ce81415a8d0d3d393230d755 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:57 UTC) from {username='oryx', principal='oryx/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:46294
I20260809 03:32:57.622575 6681 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:32:57.622774 6681 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:57.622843 7456 tablet_replica.cc:330] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27: stopping tablet replica
I20260809 03:32:57.622839 7071 tablet_service.cc:1514] Processing DeleteTablet for tablet a6645ce3ce81415a8d0d3d393230d755 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:57 UTC) from {username='oryx', principal='oryx/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:43316
I20260809 03:32:57.622951 7456 raft_consensus.cc:2229] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:57.623488 7456 raft_consensus.cc:2258] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:57.624079 7456 ts_tablet_manager.cc:1905] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:57.623658 7455 tablet_replica.cc:330] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454: stopping tablet replica
I20260809 03:32:57.624796 7455 raft_consensus.cc:2229] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:57.624961 6888 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3c221ddd9341748b688a337e0b184b (DEFAULT_TABLE table=test-table [id=5186bc0c302440f0a76382ed99af8ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:57.624990 7455 raft_consensus.cc:2258] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:57.625099 6888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3c221ddd9341748b688a337e0b184b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:57.625077 7261 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3c221ddd9341748b688a337e0b184b (DEFAULT_TABLE table=test-table [id=5186bc0c302440f0a76382ed99af8ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:57.625187 7071 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3c221ddd9341748b688a337e0b184b (DEFAULT_TABLE table=test-table [id=5186bc0c302440f0a76382ed99af8ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:57.625272 7261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3c221ddd9341748b688a337e0b184b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:57.625298 7071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3c221ddd9341748b688a337e0b184b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:57.627313 7422 tablet_bootstrap.cc:492] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27: Bootstrap starting.
I20260809 03:32:57.627362 7427 tablet_bootstrap.cc:492] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454: Bootstrap starting.
I20260809 03:32:57.628100 7422 tablet_bootstrap.cc:654] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:57.628125 7427 tablet_bootstrap.cc:654] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:57.628826 7421 tablet_bootstrap.cc:492] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995: Bootstrap starting.
I20260809 03:32:57.629020 7422 tablet_bootstrap.cc:492] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27: No bootstrap required, opened a new log
I20260809 03:32:57.629088 7422 ts_tablet_manager.cc:1397] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 03:32:57.630131 7421 tablet_bootstrap.cc:654] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:57.629482 7422 raft_consensus.cc:348] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:57.630189 7422 raft_consensus.cc:374] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:57.630244 7422 raft_consensus.cc:729] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a8cee434324accb84886ef0e91cd27, State: Initialized, Role: FOLLOWER
I20260809 03:32:57.630314 7422 consensus_queue.cc:260] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:57.630472 7422 ts_tablet_manager.cc:1428] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:57.631285 7427 tablet_bootstrap.cc:492] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454: No bootstrap required, opened a new log
I20260809 03:32:57.631338 7427 ts_tablet_manager.cc:1397] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260809 03:32:57.631368 7455 ts_tablet_manager.cc:1905] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:57.631749 7427 raft_consensus.cc:348] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:57.631830 7427 raft_consensus.cc:374] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:57.631853 7427 raft_consensus.cc:729] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4a589170ce4831801b40e9a9bcf454, State: Initialized, Role: FOLLOWER
I20260809 03:32:57.631915 7427 consensus_queue.cc:260] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:57.632056 7427 ts_tablet_manager.cc:1428] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 03:32:57.634176 7421 tablet_bootstrap.cc:492] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995: No bootstrap required, opened a new log
I20260809 03:32:57.634187 7456 ts_tablet_manager.cc:1918] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:57.634222 7421 ts_tablet_manager.cc:1397] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 03:32:57.634270 7456 log.cc:1196] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/a6645ce3ce81415a8d0d3d393230d755
I20260809 03:32:57.634392 7421 raft_consensus.cc:348] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:57.634447 7421 raft_consensus.cc:374] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:57.634461 7421 raft_consensus.cc:729] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4a9769605c4902b987b27d1d20c995, State: Initialized, Role: FOLLOWER
I20260809 03:32:57.634531 7421 consensus_queue.cc:260] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:57.634588 7455 ts_tablet_manager.cc:1918] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:57.634670 7421 ts_tablet_manager.cc:1428] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:57.634766 7455 log.cc:1196] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/wal/wals/a6645ce3ce81415a8d0d3d393230d755
I20260809 03:32:57.635332 7455 ts_tablet_manager.cc:1939] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a589170ce4831801b40e9a9bcf454: Deleting consensus metadata
I20260809 03:32:57.636206 6662 catalog_manager.cc:4841] TS 1e4a589170ce4831801b40e9a9bcf454 (127.27.40.129:42023): tablet a6645ce3ce81415a8d0d3d393230d755 (table test-table [id=e13822dd0833417c8d24978376bd383e]) successfully deleted
I20260809 03:32:57.637521 7456 ts_tablet_manager.cc:1939] T a6645ce3ce81415a8d0d3d393230d755 P 25a8cee434324accb84886ef0e91cd27: Deleting consensus metadata
I20260809 03:32:57.638053 7457 tablet_replica.cc:330] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995: stopping tablet replica
I20260809 03:32:57.638088 6666 catalog_manager.cc:4841] TS 25a8cee434324accb84886ef0e91cd27 (127.27.40.131:42861): tablet a6645ce3ce81415a8d0d3d393230d755 (table test-table [id=e13822dd0833417c8d24978376bd383e]) successfully deleted
I20260809 03:32:57.638206 7457 raft_consensus.cc:2229] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:32:57.638382 7457 raft_consensus.cc:2258] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:57.638856 7457 ts_tablet_manager.cc:1905] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:57.640834 7457 ts_tablet_manager.cc:1918] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:57.640934 7457 log.cc:1196] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/a6645ce3ce81415a8d0d3d393230d755
I20260809 03:32:57.641201 7457 ts_tablet_manager.cc:1939] T a6645ce3ce81415a8d0d3d393230d755 P 1e4a9769605c4902b987b27d1d20c995: Deleting consensus metadata
I20260809 03:32:57.641664 6660 catalog_manager.cc:4841] TS 1e4a9769605c4902b987b27d1d20c995 (127.27.40.130:32837): tablet a6645ce3ce81415a8d0d3d393230d755 (table test-table [id=e13822dd0833417c8d24978376bd383e]) successfully deleted
I20260809 03:32:57.867992 7434 raft_consensus.cc:482] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:57.868095 7434 raft_consensus.cc:504] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:57.868431 7434 leader_election.cc:288] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e4a9769605c4902b987b27d1d20c995 (127.27.40.130:32837), 25a8cee434324accb84886ef0e91cd27 (127.27.40.131:42861)
I20260809 03:32:57.871560 7420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.868707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:35679 (local address 127.27.40.131:42861)
0809 03:32:57.868776 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.868778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.868910 (+ 132us) server_negotiation.cc:407] Connection header received
0809 03:32:57.868991 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.868993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.869009 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.869072 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:57.869241 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.869786 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.870801 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.871173 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.871288 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.871300 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:57.871457 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 03:32:57.871493 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 03:32:57.871582 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.868620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:42861 (local address 127.27.40.129:35679)
0809 03:32:57.868831 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:57.868845 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:57.868923 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:57.869078 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:57.869079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:57.869081 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:57.869165 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.869167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.869873 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.869874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.870696 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.870699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.871288 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.871289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.871367 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.871370 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:32:57.871437 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 03:32:57.871481 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":3,"thread_start_us":111,"threads_started":1}
I20260809 03:32:57.871640 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.868708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:44315 (local address 127.27.40.130:32837)
0809 03:32:57.868788 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.868791 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.868917 (+ 126us) server_negotiation.cc:407] Connection header received
0809 03:32:57.868987 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.868988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.869007 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.869079 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:57.869228 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.869789 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.870914 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.871284 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.871376 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.871385 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:57.871526 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 03:32:57.871566 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 03:32:57.871815 7462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.868639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:32837 (local address 127.27.40.129:44315)
0809 03:32:57.868827 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:57.868840 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:57.868925 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:57.869082 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:57.869082 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:57.869084 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:57.869155 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.869157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.869897 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.869898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.870808 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.870810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.871378 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.871379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.871447 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.871451 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:57.871520 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 03:32:57.871562 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260809 03:32:57.871922 7317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3c221ddd9341748b688a337e0b184b" candidate_uuid: "1e4a589170ce4831801b40e9a9bcf454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25a8cee434324accb84886ef0e91cd27" is_pre_election: true
I20260809 03:32:57.872025 7317 raft_consensus.cc:2454] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e4a589170ce4831801b40e9a9bcf454 in term 0.
I20260809 03:32:57.872092 7106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3c221ddd9341748b688a337e0b184b" candidate_uuid: "1e4a589170ce4831801b40e9a9bcf454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4a9769605c4902b987b27d1d20c995" is_pre_election: true
I20260809 03:32:57.872241 6817 leader_election.cc:302] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e4a589170ce4831801b40e9a9bcf454, 25a8cee434324accb84886ef0e91cd27; no voters:
I20260809 03:32:57.872291 7106 raft_consensus.cc:2454] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e4a589170ce4831801b40e9a9bcf454 in term 0.
I20260809 03:32:57.872363 7434 raft_consensus.cc:2781] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:57.872412 7434 raft_consensus.cc:482] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:57.872427 7434 raft_consensus.cc:3037] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:57.873492 7434 raft_consensus.cc:504] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:57.873677 7434 leader_election.cc:288] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [CANDIDATE]: Term 1 election: Requested vote from peers 1e4a9769605c4902b987b27d1d20c995 (127.27.40.130:32837), 25a8cee434324accb84886ef0e91cd27 (127.27.40.131:42861)
I20260809 03:32:57.873894 7106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3c221ddd9341748b688a337e0b184b" candidate_uuid: "1e4a589170ce4831801b40e9a9bcf454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4a9769605c4902b987b27d1d20c995"
I20260809 03:32:57.873924 7317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3c221ddd9341748b688a337e0b184b" candidate_uuid: "1e4a589170ce4831801b40e9a9bcf454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25a8cee434324accb84886ef0e91cd27"
I20260809 03:32:57.874007 7317 raft_consensus.cc:3037] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:57.874013 7106 raft_consensus.cc:3037] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:57.874774 7317 raft_consensus.cc:2454] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e4a589170ce4831801b40e9a9bcf454 in term 1.
I20260809 03:32:57.874946 6817 leader_election.cc:302] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e4a589170ce4831801b40e9a9bcf454, 25a8cee434324accb84886ef0e91cd27; no voters:
I20260809 03:32:57.875036 7434 raft_consensus.cc:2781] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:57.875036 7106 raft_consensus.cc:2454] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e4a589170ce4831801b40e9a9bcf454 in term 1.
I20260809 03:32:57.875204 7434 raft_consensus.cc:686] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [term 1 LEADER]: Becoming Leader. State: Replica: 1e4a589170ce4831801b40e9a9bcf454, State: Running, Role: LEADER
I20260809 03:32:57.875303 7434 consensus_queue.cc:237] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:57.876159 6681 catalog_manager.cc:5495] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e4a589170ce4831801b40e9a9bcf454 (127.27.40.129). New cstate: current_term: 1 leader_uuid: "1e4a589170ce4831801b40e9a9bcf454" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } health_report { overall_health: UNKNOWN } } }
Aug 09 03:32:57 dist-test-slave-gkw7 krb5kdc[6616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246376, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.190@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM
I20260809 03:32:57.946485 7475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.938464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:33809 (local address 127.0.0.1:49412)
0809 03:32:57.938748 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:57.938762 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:57.938876 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:57.939122 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:57.939123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:57.939578 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:32:57.939633 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.939635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.940420 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.940421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.940993 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.940995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.941402 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.941403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.941459 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.942092 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:32:57.942097 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:32:57.943578 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:32:57.945216 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:57.945225 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:57.945228 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:57.945478 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:57.945782 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:32:57.945783 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:32:57.945783 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:32:57.945870 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:32:57.946109 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:32:57.946118 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:32:57.946164 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 03:32:57.946241 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 03:32:57.946364 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":123,"threads_started":1}
I20260809 03:32:57.946496 7476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.938551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49412 (local address 127.27.40.190:33809)
0809 03:32:57.938744 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.938747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.938852 (+ 105us) server_negotiation.cc:407] Connection header received
0809 03:32:57.938979 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.938980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.938995 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.939117 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:57.939746 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.940323 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.941112 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.941307 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.941401 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.943689 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:57.943690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:57.943691 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:57.943698 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:57.945101 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:57.945644 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:57.945645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:57.945646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:57.945712 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:57.945921 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:57.945922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:57.945922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:57.946036 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:57.946108 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:57.946281 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 03:32:57.946416 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":164,"thread_start_us":93,"threads_started":1}
I20260809 03:32:57.948534 6681 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:49412:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 03:32:57.948752 6681 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:57.950389 7071 tablet_service.cc:1467] Processing CreateTablet for tablet ee3f79fa05fc4c5690c4c08a262a2fda (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cb7ba0f310443f19bf78fcaffce4354]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:32:57.950389 7261 tablet_service.cc:1467] Processing CreateTablet for tablet ee3f79fa05fc4c5690c4c08a262a2fda (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cb7ba0f310443f19bf78fcaffce4354]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:32:57.950409 6888 tablet_service.cc:1467] Processing CreateTablet for tablet ee3f79fa05fc4c5690c4c08a262a2fda (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cb7ba0f310443f19bf78fcaffce4354]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:32:57.950565 7071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee3f79fa05fc4c5690c4c08a262a2fda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:57.950565 7261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee3f79fa05fc4c5690c4c08a262a2fda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:57.950587 6888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee3f79fa05fc4c5690c4c08a262a2fda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:57.951820 7427 tablet_bootstrap.cc:492] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454: Bootstrap starting.
I20260809 03:32:57.952363 7421 tablet_bootstrap.cc:492] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995: Bootstrap starting.
I20260809 03:32:57.952432 7422 tablet_bootstrap.cc:492] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27: Bootstrap starting.
I20260809 03:32:57.952675 7421 tablet_bootstrap.cc:654] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:57.952766 7427 tablet_bootstrap.cc:654] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:57.953253 7422 tablet_bootstrap.cc:654] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:57.953764 7427 tablet_bootstrap.cc:492] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454: No bootstrap required, opened a new log
I20260809 03:32:57.953828 7427 ts_tablet_manager.cc:1397] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 03:32:57.954003 7422 tablet_bootstrap.cc:492] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27: No bootstrap required, opened a new log
I20260809 03:32:57.954017 7427 raft_consensus.cc:348] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } }
I20260809 03:32:57.954066 7422 ts_tablet_manager.cc:1397] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:32:57.954110 7427 raft_consensus.cc:374] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:57.954131 7427 raft_consensus.cc:729] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4a589170ce4831801b40e9a9bcf454, State: Initialized, Role: FOLLOWER
I20260809 03:32:57.954157 7421 tablet_bootstrap.cc:492] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995: No bootstrap required, opened a new log
I20260809 03:32:57.954207 7421 ts_tablet_manager.cc:1397] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:32:57.954187 7427 consensus_queue.cc:260] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } }
I20260809 03:32:57.954227 7422 raft_consensus.cc:348] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } }
I20260809 03:32:57.954290 7427 ts_tablet_manager.cc:1428] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:57.954305 7422 raft_consensus.cc:374] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:57.954336 7422 raft_consensus.cc:729] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a8cee434324accb84886ef0e91cd27, State: Initialized, Role: FOLLOWER
I20260809 03:32:57.954310 7421 raft_consensus.cc:348] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } }
I20260809 03:32:57.954382 7421 raft_consensus.cc:374] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:57.954408 7421 raft_consensus.cc:729] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4a9769605c4902b987b27d1d20c995, State: Initialized, Role: FOLLOWER
I20260809 03:32:57.954381 7422 consensus_queue.cc:260] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } }
I20260809 03:32:57.954407 7434 tablet_replica.cc:439] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } }
I20260809 03:32:57.954464 7421 consensus_queue.cc:260] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } }
I20260809 03:32:57.954487 7434 tablet_replica.cc:442] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:57.954494 7422 ts_tablet_manager.cc:1428] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:57.954566 7421 ts_tablet_manager.cc:1428] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:57.954744 7433 tablet_replica.cc:439] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } }
I20260809 03:32:57.954846 7433 tablet_replica.cc:442] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:57.954962 7449 tablet_replica.cc:439] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } }
I20260809 03:32:57.955025 7449 tablet_replica.cc:442] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:57.981091 7449 raft_consensus.cc:482] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:57.981199 7449 raft_consensus.cc:504] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } }
I20260809 03:32:57.981367 7449 leader_election.cc:288] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e4a589170ce4831801b40e9a9bcf454 (127.27.40.129:42023), 25a8cee434324accb84886ef0e91cd27 (127.27.40.131:42861)
I20260809 03:32:57.981597 7317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3f79fa05fc4c5690c4c08a262a2fda" candidate_uuid: "1e4a9769605c4902b987b27d1d20c995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25a8cee434324accb84886ef0e91cd27" is_pre_election: true
I20260809 03:32:57.981621 6912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3f79fa05fc4c5690c4c08a262a2fda" candidate_uuid: "1e4a9769605c4902b987b27d1d20c995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4a589170ce4831801b40e9a9bcf454" is_pre_election: true
I20260809 03:32:57.981729 7317 raft_consensus.cc:2454] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e4a9769605c4902b987b27d1d20c995 in term 0.
I20260809 03:32:57.981724 6912 raft_consensus.cc:2454] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e4a9769605c4902b987b27d1d20c995 in term 0.
I20260809 03:32:57.981920 7007 leader_election.cc:302] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e4a589170ce4831801b40e9a9bcf454, 1e4a9769605c4902b987b27d1d20c995; no voters:
I20260809 03:32:57.982054 7449 raft_consensus.cc:2781] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:57.982107 7449 raft_consensus.cc:482] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:57.982124 7449 raft_consensus.cc:3037] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:57.982955 7449 raft_consensus.cc:504] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } }
I20260809 03:32:57.983124 7449 leader_election.cc:288] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [CANDIDATE]: Term 1 election: Requested vote from peers 1e4a589170ce4831801b40e9a9bcf454 (127.27.40.129:42023), 25a8cee434324accb84886ef0e91cd27 (127.27.40.131:42861)
I20260809 03:32:57.983312 6912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3f79fa05fc4c5690c4c08a262a2fda" candidate_uuid: "1e4a9769605c4902b987b27d1d20c995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4a589170ce4831801b40e9a9bcf454"
I20260809 03:32:57.983321 7317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3f79fa05fc4c5690c4c08a262a2fda" candidate_uuid: "1e4a9769605c4902b987b27d1d20c995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25a8cee434324accb84886ef0e91cd27"
I20260809 03:32:57.983402 7317 raft_consensus.cc:3037] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:57.983417 6912 raft_consensus.cc:3037] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:57.984081 6912 raft_consensus.cc:2454] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e4a9769605c4902b987b27d1d20c995 in term 1.
I20260809 03:32:57.984107 7317 raft_consensus.cc:2454] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e4a9769605c4902b987b27d1d20c995 in term 1.
I20260809 03:32:57.984299 7010 leader_election.cc:302] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e4a9769605c4902b987b27d1d20c995, 25a8cee434324accb84886ef0e91cd27; no voters:
I20260809 03:32:57.984411 7449 raft_consensus.cc:2781] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:57.984467 7449 raft_consensus.cc:686] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [term 1 LEADER]: Becoming Leader. State: Replica: 1e4a9769605c4902b987b27d1d20c995, State: Running, Role: LEADER
I20260809 03:32:57.984520 7449 consensus_queue.cc:237] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } }
I20260809 03:32:57.984843 7430 tablet_replica.cc:439] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995: TxnStatusTablet state changed. Reason: New leader 1e4a9769605c4902b987b27d1d20c995. Latest consensus state: current_term: 1 leader_uuid: "1e4a9769605c4902b987b27d1d20c995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } }
I20260809 03:32:57.984947 7430 tablet_replica.cc:442] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:32:57.985110 7487 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 03:32:57.985252 7487 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 03:32:57.985169 6681 catalog_manager.cc:5495] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e4a9769605c4902b987b27d1d20c995 (127.27.40.130). New cstate: current_term: 1 leader_uuid: "1e4a9769605c4902b987b27d1d20c995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } health_report { overall_health: HEALTHY } } }
I20260809 03:32:57.993412 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.990800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43340 (local address 127.27.40.130:32837)
0809 03:32:57.990868 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.990870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.990881 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:32:57.990973 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.990975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.990988 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.991050 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:57.991181 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.991716 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.992583 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.992806 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.992890 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.993094 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:57.993148 (+ 54us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:57.993223 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:57.993303 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 03:32:57.993328 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:32:57.993449 7475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.990734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:32837 (local address 127.0.0.1:43340)
0809 03:32:57.990834 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:57.990847 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:57.990902 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:57.991059 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:57.991060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:57.991061 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:32:57.991114 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.991117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.991811 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.991812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.992478 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:57.992480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.992903 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:57.992904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:57.992983 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:57.993000 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:32:57.993224 (+ 224us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:32:57.993236 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 03:32:57.993285 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 03:32:57.993342 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 03:32:57.994580 6912 raft_consensus.cc:1261] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454 [term 1 FOLLOWER]: Refusing update from remote peer 1e4a9769605c4902b987b27d1d20c995: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:57.994577 7317 raft_consensus.cc:1261] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Refusing update from remote peer 1e4a9769605c4902b987b27d1d20c995: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:57.994829 7430 consensus_queue.cc:1035] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:57.994956 7449 consensus_queue.cc:1035] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:57.995550 7433 tablet_replica.cc:439] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27: TxnStatusTablet state changed. Reason: New leader 1e4a9769605c4902b987b27d1d20c995. Latest consensus state: current_term: 1 leader_uuid: "1e4a9769605c4902b987b27d1d20c995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } }
I20260809 03:32:57.995671 7433 tablet_replica.cc:442] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:57.995608 7434 tablet_replica.cc:439] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454: TxnStatusTablet state changed. Reason: New leader 1e4a9769605c4902b987b27d1d20c995. Latest consensus state: current_term: 1 leader_uuid: "1e4a9769605c4902b987b27d1d20c995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } }
I20260809 03:32:57.995704 7434 tablet_replica.cc:442] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:57.996541 7449 tablet_replica.cc:439] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e4a9769605c4902b987b27d1d20c995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } }
I20260809 03:32:57.996651 7449 tablet_replica.cc:442] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:32:57.996718 7449 tablet_replica.cc:439] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e4a9769605c4902b987b27d1d20c995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } }
I20260809 03:32:57.996698 7446 mvcc.cc:205] Tried to move back new op lower bound from 7316465164263972864 to 7316465164224921600. Current Snapshot: MvccSnapshot[applied={T|T < 7316465164263972864}]
I20260809 03:32:57.996766 7449 tablet_replica.cc:442] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a9769605c4902b987b27d1d20c995: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:32:57.996740 7444 mvcc.cc:205] Tried to move back new op lower bound from 7316465164263972864 to 7316465164224921600. Current Snapshot: MvccSnapshot[applied={T|T < 7316465164263972864}]
I20260809 03:32:57.996841 7434 tablet_replica.cc:439] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e4a9769605c4902b987b27d1d20c995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } }
I20260809 03:32:57.996853 7433 tablet_replica.cc:439] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e4a9769605c4902b987b27d1d20c995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } }
I20260809 03:32:57.996907 7434 tablet_replica.cc:442] T ee3f79fa05fc4c5690c4c08a262a2fda P 1e4a589170ce4831801b40e9a9bcf454: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:57.996927 7433 tablet_replica.cc:442] T ee3f79fa05fc4c5690c4c08a262a2fda P 25a8cee434324accb84886ef0e91cd27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:32:58.001298 7418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:57.998365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46300 (local address 127.27.40.129:42023)
0809 03:32:57.998541 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:57.998543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:57.998548 (+ 5us) server_negotiation.cc:407] Connection header received
0809 03:32:57.998577 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:57.998578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:57.998595 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:57.998676 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:57.998822 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:57.999479 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.000413 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.000648 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.000733 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:58.000921 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:58.001044 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:58.001134 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:58.001192 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 03:32:58.001214 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260809 03:32:58.007263 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:58.004336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:56433 (local address 127.27.40.130:32837)
0809 03:32:58.004391 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:58.004393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:58.004504 (+ 111us) server_negotiation.cc:407] Connection header received
0809 03:32:58.004578 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:58.004579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:58.004593 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:58.004662 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:58.004866 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.005465 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.006458 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.006688 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.006782 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:58.006982 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:58.007021 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:58.007088 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:58.007166 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 03:32:58.007188 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 03:32:58.007529 7496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:58.004261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:32837 (local address 127.27.40.129:56433)
0809 03:32:58.004421 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:58.004433 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:58.004521 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:58.004668 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:58.004673 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:58.004674 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:32:58.004787 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:58.004789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.005571 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.005573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:58.006357 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:58.006360 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.006785 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.006787 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:58.006858 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:58.006881 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:32:58.007089 (+ 208us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:32:58.007092 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:32:58.007161 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 03:32:58.007205 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260809 03:32:58.009949 7317 raft_consensus.cc:1261] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Refusing update from remote peer 1e4a589170ce4831801b40e9a9bcf454: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:58.010023 7106 raft_consensus.cc:1261] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Refusing update from remote peer 1e4a589170ce4831801b40e9a9bcf454: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:58.010360 7464 consensus_queue.cc:1035] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:58.010501 7434 consensus_queue.cc:1035] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:58.016352 7443 mvcc.cc:205] Tried to move back new op lower bound from 7316465164327063552 to 7316465163777998848. Current Snapshot: MvccSnapshot[applied={T|T < 7316465164327063552}]
I20260809 03:32:58.022651 7418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:58.019469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:35389 (local address 127.27.40.129:42023)
0809 03:32:58.019508 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:58.019510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:58.019714 (+ 204us) server_negotiation.cc:407] Connection header received
0809 03:32:58.019800 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:58.019801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:58.019813 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:58.019879 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:58.020025 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.020598 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.021848 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.022061 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.022145 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:58.022371 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:58.022413 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:58.022442 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:58.022548 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 03:32:58.022573 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:32:58.022903 7503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:58.019433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:42023 (local address 127.27.40.130:35389)
0809 03:32:58.019648 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:58.019659 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:58.019740 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:58.019885 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:58.019886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:58.019888 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:32:58.019940 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:58.019941 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.020701 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.020702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:58.021741 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:58.021744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.022172 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.022174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:58.022239 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:58.022256 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:32:58.022484 (+ 228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:32:58.022486 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 03:32:58.022775 (+ 289us) client_negotiation.cc:240] Negotiation successful
0809 03:32:58.022827 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":100,"threads_started":1}
I20260809 03:32:58.030256 6681 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:58.030377 6681 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:58.031026 6681 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949: Sending DeleteTablet for 3 replicas of tablet 0f3c221ddd9341748b688a337e0b184b
I20260809 03:32:58.031409 7261 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3c221ddd9341748b688a337e0b184b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:58 UTC) from {username='oryx', principal='oryx/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:33800
I20260809 03:32:58.031508 7456 tablet_replica.cc:330] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27: stopping tablet replica
I20260809 03:32:58.031553 7456 raft_consensus.cc:2229] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:58.031584 7456 raft_consensus.cc:2258] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:58.031740 6888 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3c221ddd9341748b688a337e0b184b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:58 UTC) from {username='oryx', principal='oryx/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:46294
I20260809 03:32:58.031847 7455 tablet_replica.cc:330] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454: stopping tablet replica
I20260809 03:32:58.031899 7455 raft_consensus.cc:2229] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:32:58.031997 7455 raft_consensus.cc:2258] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:58.032361 7455 ts_tablet_manager.cc:1905] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:58.034322 7455 ts_tablet_manager.cc:1918] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:32:58.034379 7455 log.cc:1196] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/wal/wals/0f3c221ddd9341748b688a337e0b184b
I20260809 03:32:58.034600 7455 ts_tablet_manager.cc:1939] T 0f3c221ddd9341748b688a337e0b184b P 1e4a589170ce4831801b40e9a9bcf454: Deleting consensus metadata
I20260809 03:32:58.035085 6662 catalog_manager.cc:4841] TS 1e4a589170ce4831801b40e9a9bcf454 (127.27.40.129:42023): tablet 0f3c221ddd9341748b688a337e0b184b (table test-table [id=5186bc0c302440f0a76382ed99af8ab9]) successfully deleted
I20260809 03:32:58.036823 7071 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3c221ddd9341748b688a337e0b184b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:58 UTC) from {username='oryx', principal='oryx/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:43316
I20260809 03:32:58.036942 7457 tablet_replica.cc:330] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995: stopping tablet replica
I20260809 03:32:58.036999 7457 raft_consensus.cc:2229] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:58.037077 7457 raft_consensus.cc:2258] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:58.037463 7457 ts_tablet_manager.cc:1905] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:58.038887 7457 ts_tablet_manager.cc:1918] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:32:58.038946 7457 log.cc:1196] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/0f3c221ddd9341748b688a337e0b184b
I20260809 03:32:58.038945 7456 ts_tablet_manager.cc:1905] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:58.039170 7457 ts_tablet_manager.cc:1939] T 0f3c221ddd9341748b688a337e0b184b P 1e4a9769605c4902b987b27d1d20c995: Deleting consensus metadata
I20260809 03:32:58.039512 6660 catalog_manager.cc:4841] TS 1e4a9769605c4902b987b27d1d20c995 (127.27.40.130:32837): tablet 0f3c221ddd9341748b688a337e0b184b (table test-table [id=5186bc0c302440f0a76382ed99af8ab9]) successfully deleted
I20260809 03:32:58.040546 7456 ts_tablet_manager.cc:1918] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:32:58.040601 7456 log.cc:1196] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/0f3c221ddd9341748b688a337e0b184b
I20260809 03:32:58.040812 7456 ts_tablet_manager.cc:1939] T 0f3c221ddd9341748b688a337e0b184b P 25a8cee434324accb84886ef0e91cd27: Deleting consensus metadata
I20260809 03:32:58.041294 6666 catalog_manager.cc:4841] TS 25a8cee434324accb84886ef0e91cd27 (127.27.40.131:42861): tablet 0f3c221ddd9341748b688a337e0b184b (table test-table [id=5186bc0c302440f0a76382ed99af8ab9]) successfully deleted
Aug 09 03:32:58 dist-test-slave-gkw7 krb5kdc[6616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 03:32:58 dist-test-slave-gkw7 krb5kdc[6616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:32:58 dist-test-slave-gkw7 krb5kdc[6616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 03:32:58.067853 7476 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:32:58.060431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49414 (local address 127.27.40.190:33809)
0809 03:32:58.060475 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:58.060478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:58.060489 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:32:58.060628 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:58.060629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:58.060655 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:58.060735 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:58.061564 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.062380 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.063512 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.063796 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.063872 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:58.067781 (+ 3909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49414: BlockingRecv error: recv got EOF from 127.0.0.1:49414 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 03:32:58 dist-test-slave-gkw7 krb5kdc[6616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM
I20260809 03:32:58.078330 7476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:58.068808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49418 (local address 127.27.40.190:33809)
0809 03:32:58.068846 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:58.068848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:58.068930 (+ 82us) server_negotiation.cc:407] Connection header received
0809 03:32:58.068998 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:58.068999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:58.069013 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:58.069078 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:32:58.069715 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.070442 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.071486 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.071783 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.071864 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:58.074881 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:32:58.074883 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:32:58.074883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:32:58.074893 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:32:58.076885 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:58.077335 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:58.077336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:58.077337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:58.077407 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:32:58.077677 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:32:58.077678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:32:58.077678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:32:58.077828 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:32:58.077941 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:58.078118 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 03:32:58.078218 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 03:32:58.080323 6681 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:32:58.080518 6681 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:58.082154 7071 tablet_service.cc:1467] Processing CreateTablet for tablet 97b15792f5e64ecd95890953012e0f7f (DEFAULT_TABLE table=test-table [id=c6e6e0ba263f46ad9bd763965c24078e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:58.082243 6888 tablet_service.cc:1467] Processing CreateTablet for tablet 97b15792f5e64ecd95890953012e0f7f (DEFAULT_TABLE table=test-table [id=c6e6e0ba263f46ad9bd763965c24078e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:58.082320 7071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97b15792f5e64ecd95890953012e0f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:58.082407 6888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97b15792f5e64ecd95890953012e0f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:58.083959 7261 tablet_service.cc:1467] Processing CreateTablet for tablet 97b15792f5e64ecd95890953012e0f7f (DEFAULT_TABLE table=test-table [id=c6e6e0ba263f46ad9bd763965c24078e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:58.084118 7261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97b15792f5e64ecd95890953012e0f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:58.085462 7422 tablet_bootstrap.cc:492] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27: Bootstrap starting.
I20260809 03:32:58.085469 7421 tablet_bootstrap.cc:492] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995: Bootstrap starting.
I20260809 03:32:58.085609 7427 tablet_bootstrap.cc:492] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454: Bootstrap starting.
I20260809 03:32:58.085948 7422 tablet_bootstrap.cc:654] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:58.086054 7427 tablet_bootstrap.cc:654] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:58.086722 7422 tablet_bootstrap.cc:492] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27: No bootstrap required, opened a new log
I20260809 03:32:58.086774 7422 ts_tablet_manager.cc:1397] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:58.087033 7422 raft_consensus.cc:348] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.087106 7422 raft_consensus.cc:374] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:58.087096 7421 tablet_bootstrap.cc:654] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:58.087131 7422 raft_consensus.cc:729] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a8cee434324accb84886ef0e91cd27, State: Initialized, Role: FOLLOWER
I20260809 03:32:58.087186 7422 consensus_queue.cc:260] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.087281 7422 ts_tablet_manager.cc:1428] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:58.087862 7427 tablet_bootstrap.cc:492] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454: No bootstrap required, opened a new log
I20260809 03:32:58.087918 7427 ts_tablet_manager.cc:1397] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260809 03:32:58.088137 7427 raft_consensus.cc:348] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.088222 7427 raft_consensus.cc:374] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:58.088246 7427 raft_consensus.cc:729] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4a589170ce4831801b40e9a9bcf454, State: Initialized, Role: FOLLOWER
I20260809 03:32:58.088292 7427 consensus_queue.cc:260] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.088399 7427 ts_tablet_manager.cc:1428] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:58.088959 7421 tablet_bootstrap.cc:492] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995: No bootstrap required, opened a new log
I20260809 03:32:58.089021 7421 ts_tablet_manager.cc:1397] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 03:32:58.089147 7421 raft_consensus.cc:348] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.089229 7421 raft_consensus.cc:374] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:58.089254 7421 raft_consensus.cc:729] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4a9769605c4902b987b27d1d20c995, State: Initialized, Role: FOLLOWER
I20260809 03:32:58.089311 7421 consensus_queue.cc:260] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.089411 7421 ts_tablet_manager.cc:1428] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:58.239280 7434 raft_consensus.cc:482] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:58.239424 7434 raft_consensus.cc:504] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.239590 7434 leader_election.cc:288] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e4a9769605c4902b987b27d1d20c995 (127.27.40.130:32837), 25a8cee434324accb84886ef0e91cd27 (127.27.40.131:42861)
I20260809 03:32:58.239871 7106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97b15792f5e64ecd95890953012e0f7f" candidate_uuid: "1e4a589170ce4831801b40e9a9bcf454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4a9769605c4902b987b27d1d20c995" is_pre_election: true
I20260809 03:32:58.239867 7317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97b15792f5e64ecd95890953012e0f7f" candidate_uuid: "1e4a589170ce4831801b40e9a9bcf454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25a8cee434324accb84886ef0e91cd27" is_pre_election: true
I20260809 03:32:58.239984 7317 raft_consensus.cc:2454] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e4a589170ce4831801b40e9a9bcf454 in term 0.
I20260809 03:32:58.239984 7106 raft_consensus.cc:2454] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e4a589170ce4831801b40e9a9bcf454 in term 0.
I20260809 03:32:58.240183 6808 leader_election.cc:302] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e4a589170ce4831801b40e9a9bcf454, 1e4a9769605c4902b987b27d1d20c995; no voters:
I20260809 03:32:58.240316 7434 raft_consensus.cc:2781] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:58.240362 7434 raft_consensus.cc:482] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:58.240382 7434 raft_consensus.cc:3037] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:58.241091 7434 raft_consensus.cc:504] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.241238 7434 leader_election.cc:288] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [CANDIDATE]: Term 1 election: Requested vote from peers 1e4a9769605c4902b987b27d1d20c995 (127.27.40.130:32837), 25a8cee434324accb84886ef0e91cd27 (127.27.40.131:42861)
I20260809 03:32:58.241410 7106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97b15792f5e64ecd95890953012e0f7f" candidate_uuid: "1e4a589170ce4831801b40e9a9bcf454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4a9769605c4902b987b27d1d20c995"
I20260809 03:32:58.241489 7106 raft_consensus.cc:3037] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:58.241451 7317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97b15792f5e64ecd95890953012e0f7f" candidate_uuid: "1e4a589170ce4831801b40e9a9bcf454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25a8cee434324accb84886ef0e91cd27"
I20260809 03:32:58.241551 7317 raft_consensus.cc:3037] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:58.242156 7106 raft_consensus.cc:2454] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e4a589170ce4831801b40e9a9bcf454 in term 1.
I20260809 03:32:58.242304 7317 raft_consensus.cc:2454] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e4a589170ce4831801b40e9a9bcf454 in term 1.
I20260809 03:32:58.242347 6808 leader_election.cc:302] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e4a589170ce4831801b40e9a9bcf454, 1e4a9769605c4902b987b27d1d20c995; no voters:
I20260809 03:32:58.242427 7434 raft_consensus.cc:2781] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:58.242486 7434 raft_consensus.cc:686] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [term 1 LEADER]: Becoming Leader. State: Replica: 1e4a589170ce4831801b40e9a9bcf454, State: Running, Role: LEADER
I20260809 03:32:58.242542 7434 consensus_queue.cc:237] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.243235 6681 catalog_manager.cc:5495] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e4a589170ce4831801b40e9a9bcf454 (127.27.40.129). New cstate: current_term: 1 leader_uuid: "1e4a589170ce4831801b40e9a9bcf454" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } health_report { overall_health: UNKNOWN } } }
I20260809 03:32:58.274775 7418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:58.272626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46302 (local address 127.27.40.129:42023)
0809 03:32:58.272704 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:58.272706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:58.272715 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:32:58.272740 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:58.272741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:58.272755 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:58.272808 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:32:58.272957 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.273444 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.274187 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.274366 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.274417 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:58.274554 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:32:58.274581 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:32:58.274627 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:58.274695 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 03:32:58.274715 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 03:32:58.275766 7106 raft_consensus.cc:1261] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Refusing update from remote peer 1e4a589170ce4831801b40e9a9bcf454: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:58.275766 7317 raft_consensus.cc:1261] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Refusing update from remote peer 1e4a589170ce4831801b40e9a9bcf454: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:58.276021 7464 consensus_queue.cc:1035] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:58.276178 7434 consensus_queue.cc:1035] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:58.279047 6681 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:58.279170 6681 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:58.279817 6681 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949: Sending DeleteTablet for 3 replicas of tablet 97b15792f5e64ecd95890953012e0f7f
I20260809 03:32:58.280059 6888 tablet_service.cc:1514] Processing DeleteTablet for tablet 97b15792f5e64ecd95890953012e0f7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:58 UTC) from {username='oryx', principal='oryx/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:46294
I20260809 03:32:58.280061 7071 tablet_service.cc:1514] Processing DeleteTablet for tablet 97b15792f5e64ecd95890953012e0f7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:58 UTC) from {username='oryx', principal='oryx/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:43316
I20260809 03:32:58.280158 7457 tablet_replica.cc:330] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995: stopping tablet replica
I20260809 03:32:58.280200 7455 tablet_replica.cc:330] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454: stopping tablet replica
I20260809 03:32:58.280189 7261 tablet_service.cc:1514] Processing DeleteTablet for tablet 97b15792f5e64ecd95890953012e0f7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:58 UTC) from {username='oryx', principal='oryx/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:33800
I20260809 03:32:58.280220 7457 raft_consensus.cc:2229] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:58.280265 7457 raft_consensus.cc:2258] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:58.280265 7455 raft_consensus.cc:2229] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:32:58.280278 7456 tablet_replica.cc:330] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27: stopping tablet replica
I20260809 03:32:58.280326 7455 raft_consensus.cc:2258] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:58.280328 7456 raft_consensus.cc:2229] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:58.280381 7456 raft_consensus.cc:2258] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:58.280699 7455 ts_tablet_manager.cc:1905] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:58.280715 7457 ts_tablet_manager.cc:1905] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:58.281561 7456 ts_tablet_manager.cc:1905] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:58.281625 6681 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:32:58.281770 6681 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:32:58.282153 7457 ts_tablet_manager.cc:1918] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:58.282212 7457 log.cc:1196] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/97b15792f5e64ecd95890953012e0f7f
I20260809 03:32:58.282339 7455 ts_tablet_manager.cc:1918] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:58.282401 7455 log.cc:1196] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-0/wal/wals/97b15792f5e64ecd95890953012e0f7f
I20260809 03:32:58.282454 7457 ts_tablet_manager.cc:1939] T 97b15792f5e64ecd95890953012e0f7f P 1e4a9769605c4902b987b27d1d20c995: Deleting consensus metadata
I20260809 03:32:58.282653 7455 ts_tablet_manager.cc:1939] T 97b15792f5e64ecd95890953012e0f7f P 1e4a589170ce4831801b40e9a9bcf454: Deleting consensus metadata
I20260809 03:32:58.283142 6662 catalog_manager.cc:4841] TS 1e4a589170ce4831801b40e9a9bcf454 (127.27.40.129:42023): tablet 97b15792f5e64ecd95890953012e0f7f (table test-table [id=c6e6e0ba263f46ad9bd763965c24078e]) successfully deleted
I20260809 03:32:58.283207 7456 ts_tablet_manager.cc:1918] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:32:58.283221 6888 tablet_service.cc:1467] Processing CreateTablet for tablet 9a1c8b6122b44236850876e0ed2779e0 (DEFAULT_TABLE table=test-table [id=22fba7abf4d74127890212c43d4e4414]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:58.283258 7456 log.cc:1196] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/97b15792f5e64ecd95890953012e0f7f
I20260809 03:32:58.283264 7261 tablet_service.cc:1467] Processing CreateTablet for tablet 9a1c8b6122b44236850876e0ed2779e0 (DEFAULT_TABLE table=test-table [id=22fba7abf4d74127890212c43d4e4414]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:58.283329 6888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a1c8b6122b44236850876e0ed2779e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:58.283378 7261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a1c8b6122b44236850876e0ed2779e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:58.283499 7456 ts_tablet_manager.cc:1939] T 97b15792f5e64ecd95890953012e0f7f P 25a8cee434324accb84886ef0e91cd27: Deleting consensus metadata
I20260809 03:32:58.284174 7427 tablet_bootstrap.cc:492] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454: Bootstrap starting.
I20260809 03:32:58.284343 6660 catalog_manager.cc:4841] TS 1e4a9769605c4902b987b27d1d20c995 (127.27.40.130:32837): tablet 97b15792f5e64ecd95890953012e0f7f (table test-table [id=c6e6e0ba263f46ad9bd763965c24078e]) successfully deleted
I20260809 03:32:58.284441 7071 tablet_service.cc:1467] Processing CreateTablet for tablet 9a1c8b6122b44236850876e0ed2779e0 (DEFAULT_TABLE table=test-table [id=22fba7abf4d74127890212c43d4e4414]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:32:58.284488 7422 tablet_bootstrap.cc:492] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27: Bootstrap starting.
I20260809 03:32:58.284516 7427 tablet_bootstrap.cc:654] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:58.284549 7071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a1c8b6122b44236850876e0ed2779e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:32:58.285040 7422 tablet_bootstrap.cc:654] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:58.285161 7427 tablet_bootstrap.cc:492] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454: No bootstrap required, opened a new log
I20260809 03:32:58.285205 7427 ts_tablet_manager.cc:1397] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 03:32:58.285322 7427 raft_consensus.cc:348] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.285386 7427 raft_consensus.cc:374] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:58.285404 7427 raft_consensus.cc:729] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4a589170ce4831801b40e9a9bcf454, State: Initialized, Role: FOLLOWER
I20260809 03:32:58.285475 7427 consensus_queue.cc:260] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.285629 7427 ts_tablet_manager.cc:1428] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:58.285728 7422 tablet_bootstrap.cc:492] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27: No bootstrap required, opened a new log
I20260809 03:32:58.285770 7422 ts_tablet_manager.cc:1397] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:58.285844 7421 tablet_bootstrap.cc:492] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995: Bootstrap starting.
I20260809 03:32:58.285874 7422 raft_consensus.cc:348] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.285917 7422 raft_consensus.cc:374] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:58.285930 7422 raft_consensus.cc:729] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a8cee434324accb84886ef0e91cd27, State: Initialized, Role: FOLLOWER
I20260809 03:32:58.285991 7422 consensus_queue.cc:260] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.286085 7422 ts_tablet_manager.cc:1428] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:32:58.286177 7421 tablet_bootstrap.cc:654] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995: Neither blocks nor log segments found. Creating new log.
I20260809 03:32:58.286547 6666 catalog_manager.cc:4841] TS 25a8cee434324accb84886ef0e91cd27 (127.27.40.131:42861): tablet 97b15792f5e64ecd95890953012e0f7f (table test-table [id=c6e6e0ba263f46ad9bd763965c24078e]) successfully deleted
I20260809 03:32:58.287123 7421 tablet_bootstrap.cc:492] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995: No bootstrap required, opened a new log
I20260809 03:32:58.287179 7421 ts_tablet_manager.cc:1397] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:32:58.287285 7421 raft_consensus.cc:348] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.287367 7421 raft_consensus.cc:374] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:32:58.287400 7421 raft_consensus.cc:729] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4a9769605c4902b987b27d1d20c995, State: Initialized, Role: FOLLOWER
I20260809 03:32:58.287453 7421 consensus_queue.cc:260] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.287546 7421 ts_tablet_manager.cc:1428] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 03:32:58.334371 7433 raft_consensus.cc:482] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:58.334504 7433 raft_consensus.cc:504] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.334933 7433 leader_election.cc:288] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e4a589170ce4831801b40e9a9bcf454 (127.27.40.129:42023), 1e4a9769605c4902b987b27d1d20c995 (127.27.40.130:32837)
I20260809 03:32:58.336922 7464 raft_consensus.cc:482] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:32:58.337013 7464 raft_consensus.cc:504] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.337167 7464 leader_election.cc:288] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e4a9769605c4902b987b27d1d20c995 (127.27.40.130:32837), 25a8cee434324accb84886ef0e91cd27 (127.27.40.131:42861)
I20260809 03:32:58.337366 7317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a1c8b6122b44236850876e0ed2779e0" candidate_uuid: "1e4a589170ce4831801b40e9a9bcf454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25a8cee434324accb84886ef0e91cd27" is_pre_election: true
I20260809 03:32:58.337373 7106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a1c8b6122b44236850876e0ed2779e0" candidate_uuid: "1e4a589170ce4831801b40e9a9bcf454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4a9769605c4902b987b27d1d20c995" is_pre_election: true
I20260809 03:32:58.337460 7317 raft_consensus.cc:2454] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e4a589170ce4831801b40e9a9bcf454 in term 0.
I20260809 03:32:58.337462 7106 raft_consensus.cc:2454] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e4a589170ce4831801b40e9a9bcf454 in term 0.
I20260809 03:32:58.337659 6808 leader_election.cc:302] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e4a589170ce4831801b40e9a9bcf454, 1e4a9769605c4902b987b27d1d20c995; no voters:
I20260809 03:32:58.337781 7464 raft_consensus.cc:2781] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:32:58.337821 7464 raft_consensus.cc:482] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:32:58.337847 7464 raft_consensus.cc:3037] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:58.338565 7418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:58.335203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:49759 (local address 127.27.40.129:42023)
0809 03:32:58.335282 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:58.335285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:58.335335 (+ 50us) server_negotiation.cc:407] Connection header received
0809 03:32:58.335441 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:58.335442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:58.335450 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:58.335511 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:58.335807 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.336296 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.337703 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.338160 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.338201 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:58.338209 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:58.338463 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 03:32:58.338499 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:32:58.338637 7536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:58.335105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:42023 (local address 127.27.40.131:49759)
0809 03:32:58.335267 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:58.335281 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:58.335355 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:58.335532 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:58.335533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:58.335535 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:58.335742 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:58.335745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.336388 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.336389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:58.337588 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:58.337590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.338274 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.338275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:58.338377 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:58.338381 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:58.338453 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 03:32:58.338495 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":36,"thread_start_us":71,"threads_started":1}
I20260809 03:32:58.338639 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:58.335197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:40475 (local address 127.27.40.130:32837)
0809 03:32:58.335269 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 03:32:58.335277 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:32:58.335398 (+ 121us) server_negotiation.cc:407] Connection header received
0809 03:32:58.335471 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:32:58.335472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:32:58.335483 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:32:58.335529 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:32:58.335708 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.336210 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.337722 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.338240 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.338289 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:58.338297 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 03:32:58.338515 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 03:32:58.338562 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 03:32:58.338905 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:58.335126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:32837 (local address 127.27.40.131:40475)
0809 03:32:58.335325 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:58.335334 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:32:58.335414 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:32:58.335545 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:32:58.335546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:32:58.335548 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:32:58.335638 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:58.335640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.336396 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.336397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:58.337625 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:32:58.337629 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:32:58.338365 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:32:58.338365 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:32:58.338435 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:32:58.338439 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:32:58.338762 (+ 323us) client_negotiation.cc:240] Negotiation successful
0809 03:32:58.338812 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
I20260809 03:32:58.338871 7464 raft_consensus.cc:504] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.338909 6912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a1c8b6122b44236850876e0ed2779e0" candidate_uuid: "25a8cee434324accb84886ef0e91cd27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4a589170ce4831801b40e9a9bcf454" is_pre_election: true
I20260809 03:32:58.338995 7464 leader_election.cc:288] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [CANDIDATE]: Term 1 election: Requested vote from peers 1e4a9769605c4902b987b27d1d20c995 (127.27.40.130:32837), 25a8cee434324accb84886ef0e91cd27 (127.27.40.131:42861)
I20260809 03:32:58.339025 6912 raft_consensus.cc:2379] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 25a8cee434324accb84886ef0e91cd27 in current term 1: Already voted for candidate 1e4a589170ce4831801b40e9a9bcf454 in this term.
I20260809 03:32:58.339231 7106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a1c8b6122b44236850876e0ed2779e0" candidate_uuid: "1e4a589170ce4831801b40e9a9bcf454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4a9769605c4902b987b27d1d20c995"
I20260809 03:32:58.339305 7106 raft_consensus.cc:3037] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:58.339278 7317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a1c8b6122b44236850876e0ed2779e0" candidate_uuid: "1e4a589170ce4831801b40e9a9bcf454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25a8cee434324accb84886ef0e91cd27"
I20260809 03:32:58.339342 7317 raft_consensus.cc:3037] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:32:58.339319 7103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a1c8b6122b44236850876e0ed2779e0" candidate_uuid: "25a8cee434324accb84886ef0e91cd27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4a9769605c4902b987b27d1d20c995" is_pre_election: true
I20260809 03:32:58.339751 7209 leader_election.cc:302] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25a8cee434324accb84886ef0e91cd27; no voters: 1e4a589170ce4831801b40e9a9bcf454, 1e4a9769605c4902b987b27d1d20c995
I20260809 03:32:58.340164 7317 raft_consensus.cc:2454] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e4a589170ce4831801b40e9a9bcf454 in term 1.
I20260809 03:32:58.340181 7106 raft_consensus.cc:2454] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e4a589170ce4831801b40e9a9bcf454 in term 1.
I20260809 03:32:58.340250 7433 raft_consensus.cc:2726] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 03:32:58.340350 6817 leader_election.cc:302] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e4a589170ce4831801b40e9a9bcf454, 25a8cee434324accb84886ef0e91cd27; no voters:
I20260809 03:32:58.340461 7464 raft_consensus.cc:2781] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:32:58.340516 7464 raft_consensus.cc:686] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [term 1 LEADER]: Becoming Leader. State: Replica: 1e4a589170ce4831801b40e9a9bcf454, State: Running, Role: LEADER
I20260809 03:32:58.340569 7464 consensus_queue.cc:237] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } }
I20260809 03:32:58.341233 6681 catalog_manager.cc:5495] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e4a589170ce4831801b40e9a9bcf454 (127.27.40.129). New cstate: current_term: 1 leader_uuid: "1e4a589170ce4831801b40e9a9bcf454" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4a589170ce4831801b40e9a9bcf454" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 } health_report { overall_health: UNKNOWN } } }
I20260809 03:32:58.362709 7317 raft_consensus.cc:1261] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Refusing update from remote peer 1e4a589170ce4831801b40e9a9bcf454: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:58.362709 7106 raft_consensus.cc:1261] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Refusing update from remote peer 1e4a589170ce4831801b40e9a9bcf454: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:32:58.363022 7532 consensus_queue.cc:1035] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e4a9769605c4902b987b27d1d20c995" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 32837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:58.363247 7464 consensus_queue.cc:1035] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25a8cee434324accb84886ef0e91cd27" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:32:58.372707 6681 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:58.372828 6681 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:32:58.373507 6681 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34abefd0f5d94500b9f64e7e2cbbb949: Sending DeleteTablet for 3 replicas of tablet 9a1c8b6122b44236850876e0ed2779e0
I20260809 03:32:58.373838 6888 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a1c8b6122b44236850876e0ed2779e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:58 UTC) from {username='oryx', principal='oryx/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:46294
I20260809 03:32:58.373847 7071 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a1c8b6122b44236850876e0ed2779e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:58 UTC) from {username='oryx', principal='oryx/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:43316
I20260809 03:32:58.373864 7261 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a1c8b6122b44236850876e0ed2779e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:32:58 UTC) from {username='oryx', principal='oryx/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:33800
I20260809 03:32:58.373935 7455 tablet_replica.cc:330] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454: stopping tablet replica
I20260809 03:32:58.373951 7457 tablet_replica.cc:330] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995: stopping tablet replica
I20260809 03:32:58.373986 7455 raft_consensus.cc:2229] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:32:58.374011 7457 raft_consensus.cc:2229] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:58.374042 7457 raft_consensus.cc:2258] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:58.374054 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 6782
I20260809 03:32:58.374044 7455 raft_consensus.cc:2258] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a589170ce4831801b40e9a9bcf454 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:58.374111 7456 tablet_replica.cc:330] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27: stopping tablet replica
I20260809 03:32:58.374161 7456 raft_consensus.cc:2229] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:32:58.374192 7456 raft_consensus.cc:2258] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:32:58.380785 7457 ts_tablet_manager.cc:1905] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:58.382090 7456 ts_tablet_manager.cc:1905] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:32:58.384003 7456 ts_tablet_manager.cc:1918] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:32:58.384056 7456 log.cc:1196] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/9a1c8b6122b44236850876e0ed2779e0
I20260809 03:32:58.384249 7456 ts_tablet_manager.cc:1939] T 9a1c8b6122b44236850876e0ed2779e0 P 25a8cee434324accb84886ef0e91cd27: Deleting consensus metadata
I20260809 03:32:58.384707 7457 ts_tablet_manager.cc:1918] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:32:58.384769 7457 log.cc:1196] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/9a1c8b6122b44236850876e0ed2779e0
I20260809 03:32:58.384899 6666 catalog_manager.cc:4841] TS 25a8cee434324accb84886ef0e91cd27 (127.27.40.131:42861): tablet 9a1c8b6122b44236850876e0ed2779e0 (table test-table [id=22fba7abf4d74127890212c43d4e4414]) successfully deleted
I20260809 03:32:58.385002 7457 ts_tablet_manager.cc:1939] T 9a1c8b6122b44236850876e0ed2779e0 P 1e4a9769605c4902b987b27d1d20c995: Deleting consensus metadata
I20260809 03:32:58.385402 6660 catalog_manager.cc:4841] TS 1e4a9769605c4902b987b27d1d20c995 (127.27.40.130:32837): tablet 9a1c8b6122b44236850876e0ed2779e0 (table test-table [id=22fba7abf4d74127890212c43d4e4414]) successfully deleted
W20260809 03:32:58.391319 6662 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.40.129:42023 (error 108)
I20260809 03:32:58.391640 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 6988
I20260809 03:32:58.393421 7546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:32:58.391781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:42023 (local address 127.0.0.1:46316)
0809 03:32:58.393334 (+ 1553us) negotiation.cc:107] Waiting for socket to connect
0809 03:32:58.393384 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:42023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1521,"thread_start_us":55,"threads_started":1}
W20260809 03:32:58.393541 6662 catalog_manager.cc:4584] TS 1e4a589170ce4831801b40e9a9bcf454 (127.27.40.129:42023): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9a1c8b6122b44236850876e0ed2779e0: Network error: Client connection negotiation failed: client connection to 127.27.40.129:42023: connect: Connection refused (error 111)
I20260809 03:32:58.408804 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 7183
I20260809 03:32:58.419225 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 6637
2026-08-09T03:32:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3353 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:32:58 dist-test-slave-gkw7 krb5kdc[7551](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:32:58 dist-test-slave-gkw7 krb5kdc[7551](info): set up 2 sockets
Aug 09 03:32:58 dist-test-slave-gkw7 krb5kdc[7551](info): commencing operation
krb5kdc: starting...
W20260809 03:32:59.756705 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:32:59 dist-test-slave-gkw7 krb5kdc[7551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:32:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:32:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:32:59.936019 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:42769
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:42769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:33:00.018076 7569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:00.018334 7569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:00.018371 7569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:00.019523 7569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:33:00.019589 7569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:00.019613 7569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:00.019635 7569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:33:00.019663 7569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:33:00.021157 7569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:42769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:42769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:00.021436 7569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:00.021696 7569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:00.026574 7583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:00.026574 7580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:00.026697 7581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:00.027078 7569 server_base.cc:1034] running on GCE node
I20260809 03:33:00.027374 7569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:00.027757 7569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:00.028940 7569 hybrid_clock.cc:648] HybridClock initialized: now 1786246380028922 us; error 27 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T03:33:00Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1606 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:33:00 dist-test-slave-gkw7 krb5kdc[7591](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:33:00 dist-test-slave-gkw7 krb5kdc[7591](info): set up 2 sockets
Aug 09 03:33:00 dist-test-slave-gkw7 krb5kdc[7591](info): commencing operation
krb5kdc: starting...
W20260809 03:33:01.260485 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:33:01 dist-test-slave-gkw7 krb5kdc[7591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:33:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:33:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:33:01.392464 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:34409
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:36933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:34409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:33:01.475905 7606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:01.476135 7606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:01.476173 7606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:01.477366 7606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:33:01.477427 7606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:01.477453 7606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:01.477485 7606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:33:01.477512 7606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:33:01.479002 7606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:36933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:34409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:34409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:01.479290 7606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:01.479509 7606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:33:01.484651 7606 server_base.cc:1034] running on GCE node
W20260809 03:33:01.484730 7615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:01.484618 7617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:01.484606 7613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:01.485131 7606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:01.485456 7606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:01.486636 7606 hybrid_clock.cc:648] HybridClock initialized: now 1786246381486633 us; error 58 us; skew 500 ppm
Aug 09 03:33:01 dist-test-slave-gkw7 krb5kdc[7591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246381, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:33:01.489189 7606 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:33:01.489753 7606 webserver.cc:460] Webserver started at http://127.27.40.190:40787/ using document root <none> and password file <none>
I20260809 03:33:01.490005 7606 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:01.490082 7606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:01.490242 7606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:33:01.491313 7606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "6f5d0838ed424e7db270114e2de7058f"
format_stamp: "Formatted at 2026-08-09 03:33:01 on dist-test-slave-gkw7"
server_key: "6fe5a7019d63ad5305f57d86f50bb8f3"
server_key_iv: "35783c830bb86b3d63d8c2e9aec01ef6"
server_key_version: "encryptionkey@0"
I20260809 03:33:01.491688 7606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "6f5d0838ed424e7db270114e2de7058f"
format_stamp: "Formatted at 2026-08-09 03:33:01 on dist-test-slave-gkw7"
server_key: "6fe5a7019d63ad5305f57d86f50bb8f3"
server_key_iv: "35783c830bb86b3d63d8c2e9aec01ef6"
server_key_version: "encryptionkey@0"
I20260809 03:33:01.493058 7606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:33:01.493922 7627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:01.494097 7606 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:33:01.494221 7606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "6f5d0838ed424e7db270114e2de7058f"
format_stamp: "Formatted at 2026-08-09 03:33:01 on dist-test-slave-gkw7"
server_key: "6fe5a7019d63ad5305f57d86f50bb8f3"
server_key_iv: "35783c830bb86b3d63d8c2e9aec01ef6"
server_key_version: "encryptionkey@0"
I20260809 03:33:01.494302 7606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:01.509030 7606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:01.511240 7606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:01.511413 7606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:01.515823 7606 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:34409
I20260809 03:33:01.515867 7707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:34409 every 8 connection(s)
I20260809 03:33:01.516263 7606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:33:01.517132 7708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:33:01.518016 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 7606
I20260809 03:33:01.518136 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:33:01.518329 27810 external_mini_cluster.cc:1428] Setting key 45cf8d2bb74987792fdf57acdf2192d9
I20260809 03:33:01.520427 7708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f: Bootstrap starting.
I20260809 03:33:01.521147 7708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f: Neither blocks nor log segments found. Creating new log.
I20260809 03:33:01.521457 7708 log.cc:824] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:33:01.522207 7708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f: No bootstrap required, opened a new log
I20260809 03:33:01.524359 7708 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5d0838ed424e7db270114e2de7058f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34409 } }
I20260809 03:33:01.524488 7708 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:33:01.524511 7708 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f5d0838ed424e7db270114e2de7058f, State: Initialized, Role: FOLLOWER
I20260809 03:33:01.524650 7708 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5d0838ed424e7db270114e2de7058f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34409 } }
I20260809 03:33:01.524725 7708 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:33:01.524747 7708 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:33:01.524813 7708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:33:01.525319 7708 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5d0838ed424e7db270114e2de7058f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34409 } }
Aug 09 03:33:01 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:33:01.525436 7708 leader_election.cc:302] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f5d0838ed424e7db270114e2de7058f; no voters:
I20260809 03:33:01.525674 7708 leader_election.cc:288] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:33:01.525710 7714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:33:01.525825 7714 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 1 LEADER]: Becoming Leader. State: Replica: 6f5d0838ed424e7db270114e2de7058f, State: Running, Role: LEADER
I20260809 03:33:01.525949 7708 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:33:01.525980 7714 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5d0838ed424e7db270114e2de7058f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34409 } }
I20260809 03:33:01.526625 7715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f5d0838ed424e7db270114e2de7058f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5d0838ed424e7db270114e2de7058f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34409 } } }
I20260809 03:33:01.526818 7716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f5d0838ed424e7db270114e2de7058f. Latest consensus state: current_term: 1 leader_uuid: "6f5d0838ed424e7db270114e2de7058f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5d0838ed424e7db270114e2de7058f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34409 } } }
I20260809 03:33:01.527218 7716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [sys.catalog]: This master's current role is: LEADER
I20260809 03:33:01.526887 7715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [sys.catalog]: This master's current role is: LEADER
W20260809 03:33:01.527904 7734 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:33:01.527962 7734 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:33:01.528317 7735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:33:01.528532 7735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:33:01.529838 7712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:01.518865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39748 (local address 127.27.40.190:34409)
0809 03:33:01.519067 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:01.519076 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:01.519109 (+ 33us) server_negotiation.cc:407] Connection header received
0809 03:33:01.519679 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:01.519689 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:01.519757 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:01.519904 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:01.520533 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:01.521908 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:01.522711 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:01.523341 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:01.523470 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:01.526036 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:01.526040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:01.526041 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:33:01.526052 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:01.528122 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:01.528555 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:01.528559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:01.528559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:01.528665 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:01.528892 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:01.528893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:01.528894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:01.529202 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:01.529353 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:01.529479 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 03:33:01.529642 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260809 03:33:01.530490 7735 catalog_manager.cc:1347] Generated new cluster ID: ef7709dd5df141db89ff14fa7172df2c
I20260809 03:33:01.530547 7735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:33:01.540510 7735 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:33:01.541020 7735 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:33:01.557261 7735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f: Generated new TSK 0
I20260809 03:33:01.557458 7735 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:33:01.596967 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34409
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:36933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:33:01.682250 7742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:01.682442 7742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:01.682466 7742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:01.683549 7742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:01.683583 7742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:01.683607 7742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:33:01.684947 7742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:36933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:01.685143 7742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:01.685329 7742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:01.690694 7756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:01.690819 7742 server_base.cc:1034] running on GCE node
W20260809 03:33:01.690917 7751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:01.690836 7750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:01.691259 7742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:01.691506 7742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:01.692680 7742 hybrid_clock.cc:648] HybridClock initialized: now 1786246381692670 us; error 48 us; skew 500 ppm
Aug 09 03:33:01 dist-test-slave-gkw7 krb5kdc[7591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246381, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:33:01.695317 7742 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:33:01.695739 7742 webserver.cc:460] Webserver started at http://127.27.40.129:46295/ using document root <none> and password file <none>
I20260809 03:33:01.695990 7742 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:01.696060 7742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:01.696218 7742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:33:01.697350 7742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "10517aad99f94fc7b8b6cdc8021a33b2"
format_stamp: "Formatted at 2026-08-09 03:33:01 on dist-test-slave-gkw7"
server_key: "e55e4e2cd3b261b55eebdbb7c7a59564"
server_key_iv: "d28889e7d9f2d3663193074fc1c49861"
server_key_version: "encryptionkey@0"
I20260809 03:33:01.697762 7742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "10517aad99f94fc7b8b6cdc8021a33b2"
format_stamp: "Formatted at 2026-08-09 03:33:01 on dist-test-slave-gkw7"
server_key: "e55e4e2cd3b261b55eebdbb7c7a59564"
server_key_iv: "d28889e7d9f2d3663193074fc1c49861"
server_key_version: "encryptionkey@0"
I20260809 03:33:01.699169 7742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:33:01.699922 7767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:01.700121 7742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:33:01.700251 7742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "10517aad99f94fc7b8b6cdc8021a33b2"
format_stamp: "Formatted at 2026-08-09 03:33:01 on dist-test-slave-gkw7"
server_key: "e55e4e2cd3b261b55eebdbb7c7a59564"
server_key_iv: "d28889e7d9f2d3663193074fc1c49861"
server_key_version: "encryptionkey@0"
I20260809 03:33:01.700326 7742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:01.707003 7742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:01.709056 7742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:01.709244 7742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:01.709899 7742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:33:01.710222 7742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:33:01.710278 7742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:01.710320 7742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:33:01.710345 7742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:01.716018 7742 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:33845
I20260809 03:33:01.716121 7927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:33845 every 8 connection(s)
I20260809 03:33:01.716411 7742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
Aug 09 03:33:01 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246381, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:33:01.722532 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 7742
I20260809 03:33:01.722672 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:33:01.722909 27810 external_mini_cluster.cc:1428] Setting key cf746406f9984b9f74c1f19ded8fbf4e
I20260809 03:33:01.725826 7712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:01.717135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:52171 (local address 127.27.40.190:34409)
0809 03:33:01.717216 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:01.717219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:01.717375 (+ 156us) server_negotiation.cc:407] Connection header received
0809 03:33:01.717912 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:01.717913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:01.717925 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:01.717978 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:01.718870 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:01.719377 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:01.720369 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:01.720567 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:01.720623 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:01.722836 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:01.722837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:01.722837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:33:01.722844 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:01.724288 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:01.724712 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:01.724714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:01.724714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:01.724776 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:01.725034 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:01.725035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:01.725035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:01.725204 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:01.725280 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:01.725597 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 03:33:01.725695 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:33:01.726083 7931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:01.717081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:52171)
0809 03:33:01.717259 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:01.717295 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:01.717825 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:01.718049 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:01.718052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:01.718589 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:01.718782 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:01.718787 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:01.719454 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:01.719454 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:01.720270 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:01.720288 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:01.720651 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:01.720652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:01.720752 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:01.721196 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:01.721204 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:01.722744 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:33:01.724377 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:33:01.724384 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:33:01.724387 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:33:01.724612 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:33:01.724845 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:33:01.724846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:33:01.724846 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:33:01.724963 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:33:01.725317 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:33:01.725324 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:33:01.725442 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 03:33:01.725600 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 03:33:01.725746 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":98,"threads_started":1}
I20260809 03:33:01.726671 7929 heartbeater.cc:347] Connected to a master server at 127.27.40.190:34409
I20260809 03:33:01.726776 7929 heartbeater.cc:464] Registering TS with master...
I20260809 03:33:01.726969 7929 heartbeater.cc:511] Master 127.27.40.190:34409 requested a full tablet report, sending...
I20260809 03:33:01.727489 7647 ts_manager.cc:194] Registered new tserver with Master: 10517aad99f94fc7b8b6cdc8021a33b2 (127.27.40.129:33845)
I20260809 03:33:01.728236 7647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:52171
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:33:01.750402 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34409
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:36933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:33:01.832419 7937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:01.832667 7937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:01.832706 7937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:01.833947 7937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:01.833985 7937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:01.834012 7937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:33:01.835482 7937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:36933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:01.835726 7937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:01.835973 7937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:01.840984 7949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:01.840997 7942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:01.841369 7945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:01.841457 7937 server_base.cc:1034] running on GCE node
I20260809 03:33:01.841710 7937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:01.841909 7937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:01.843047 7937 hybrid_clock.cc:648] HybridClock initialized: now 1786246381843036 us; error 27 us; skew 500 ppm
Aug 09 03:33:01 dist-test-slave-gkw7 krb5kdc[7591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246381, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:33:01.845715 7937 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:33:01.846125 7937 webserver.cc:460] Webserver started at http://127.27.40.130:38343/ using document root <none> and password file <none>
I20260809 03:33:01.846354 7937 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:01.846398 7937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:01.846571 7937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:33:01.847605 7937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "eb049bed7df042b3874f6c3082dcb459"
format_stamp: "Formatted at 2026-08-09 03:33:01 on dist-test-slave-gkw7"
server_key: "472c07f40386503a853627a28368f979"
server_key_iv: "c54876aa1a06058eea721c302f41b38f"
server_key_version: "encryptionkey@0"
I20260809 03:33:01.847960 7937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "eb049bed7df042b3874f6c3082dcb459"
format_stamp: "Formatted at 2026-08-09 03:33:01 on dist-test-slave-gkw7"
server_key: "472c07f40386503a853627a28368f979"
server_key_iv: "c54876aa1a06058eea721c302f41b38f"
server_key_version: "encryptionkey@0"
I20260809 03:33:01.849272 7937 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:33:01.850117 7958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:01.850335 7937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 03:33:01.850409 7937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "eb049bed7df042b3874f6c3082dcb459"
format_stamp: "Formatted at 2026-08-09 03:33:01 on dist-test-slave-gkw7"
server_key: "472c07f40386503a853627a28368f979"
server_key_iv: "c54876aa1a06058eea721c302f41b38f"
server_key_version: "encryptionkey@0"
I20260809 03:33:01.850487 7937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:01.869931 7937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:01.872252 7937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:01.872424 7937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:01.872650 7937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:33:01.872953 7937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:33:01.872987 7937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:01.873008 7937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:33:01.873054 7937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:01.878839 7937 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:32887
I20260809 03:33:01.878886 8119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:32887 every 8 connection(s)
I20260809 03:33:01.879191 7937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
Aug 09 03:33:01 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246381, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:33:01.885722 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 7937
I20260809 03:33:01.885836 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:33:01.886031 27810 external_mini_cluster.cc:1428] Setting key 6d062dde29ac7a10af1c0d88a942d353
I20260809 03:33:01.888420 7712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:01.879845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:58195 (local address 127.27.40.190:34409)
0809 03:33:01.879892 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:01.879894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:01.880098 (+ 204us) server_negotiation.cc:407] Connection header received
0809 03:33:01.880553 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:01.880555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:01.880565 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:01.880618 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:01.881480 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:01.882049 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:01.882990 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:01.883184 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:01.883234 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:01.885347 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:01.885349 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:01.885349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:33:01.885356 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:01.887050 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:01.887445 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:01.887446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:01.887446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:01.887513 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:01.887765 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:01.887766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:01.887767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:01.887888 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:01.887969 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:01.888231 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 03:33:01.888319 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 03:33:01.888645 8123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:01.879828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:58195)
0809 03:33:01.879976 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:01.880010 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:01.880471 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:01.880666 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:01.880669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:01.881218 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:01.881404 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:01.881409 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:01.882131 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:01.882132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:01.882918 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:01.882920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:01.883257 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:01.883258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:01.883354 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:01.883863 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:01.883871 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:01.885231 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:33:01.887140 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:33:01.887147 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:33:01.887149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:33:01.887365 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:33:01.887588 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:33:01.887589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:33:01.887589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:33:01.887694 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:33:01.887958 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:33:01.887964 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:33:01.888099 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 03:33:01.888211 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 03:33:01.888399 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":90,"threads_started":1}
I20260809 03:33:01.889302 8120 heartbeater.cc:347] Connected to a master server at 127.27.40.190:34409
I20260809 03:33:01.889384 8120 heartbeater.cc:464] Registering TS with master...
I20260809 03:33:01.889647 8120 heartbeater.cc:511] Master 127.27.40.190:34409 requested a full tablet report, sending...
I20260809 03:33:01.890087 7647 ts_manager.cc:194] Registered new tserver with Master: eb049bed7df042b3874f6c3082dcb459 (127.27.40.130:32887)
I20260809 03:33:01.890514 7647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:58195
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:33:01.914561 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34409
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:36933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:33:01.999559 8127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:01.999779 8127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:01.999804 8127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:02.001066 8127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:02.001109 8127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:02.001157 8127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:33:02.002933 8127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:36933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:02.003263 8127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:02.003515 8127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:02.008843 8137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:02.008862 8140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:02.008999 8136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:02.009109 8127 server_base.cc:1034] running on GCE node
I20260809 03:33:02.009338 8127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:02.009675 8127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:02.010851 8127 hybrid_clock.cc:648] HybridClock initialized: now 1786246382010827 us; error 38 us; skew 500 ppm
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246382, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:33:02.013797 8127 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:33:02.014340 8127 webserver.cc:460] Webserver started at http://127.27.40.131:45357/ using document root <none> and password file <none>
I20260809 03:33:02.014592 8127 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:02.014674 8127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:02.014837 8127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:33:02.015966 8127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "2ea483a1ca1c481da25b7e46cfbc11ed"
format_stamp: "Formatted at 2026-08-09 03:33:02 on dist-test-slave-gkw7"
server_key: "4b0405112935c5e6c2ddf8d1e8ed59b1"
server_key_iv: "23387d9df3d82f0962b557ba505f865d"
server_key_version: "encryptionkey@0"
I20260809 03:33:02.016351 8127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "2ea483a1ca1c481da25b7e46cfbc11ed"
format_stamp: "Formatted at 2026-08-09 03:33:02 on dist-test-slave-gkw7"
server_key: "4b0405112935c5e6c2ddf8d1e8ed59b1"
server_key_iv: "23387d9df3d82f0962b557ba505f865d"
server_key_version: "encryptionkey@0"
I20260809 03:33:02.017858 8127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:33:02.018659 8150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:02.018846 8127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:33:02.018954 8127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "2ea483a1ca1c481da25b7e46cfbc11ed"
format_stamp: "Formatted at 2026-08-09 03:33:02 on dist-test-slave-gkw7"
server_key: "4b0405112935c5e6c2ddf8d1e8ed59b1"
server_key_iv: "23387d9df3d82f0962b557ba505f865d"
server_key_version: "encryptionkey@0"
I20260809 03:33:02.019030 8127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:02.032788 8127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:02.035326 8127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:02.035493 8127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:02.035724 8127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:33:02.036038 8127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:33:02.036093 8127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:02.036134 8127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:33:02.036162 8127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:02.042029 8127 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:39405
I20260809 03:33:02.042073 8307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:39405 every 8 connection(s)
I20260809 03:33:02.042433 8127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246382, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:33:02.050323 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 8127
I20260809 03:33:02.050455 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:33:02.050698 27810 external_mini_cluster.cc:1428] Setting key 612e2f3b031fefcce8f7d2fbc2c7739b
I20260809 03:33:02.052060 7712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:02.043182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:53359 (local address 127.27.40.190:34409)
0809 03:33:02.043232 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:02.043235 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:02.043414 (+ 179us) server_negotiation.cc:407] Connection header received
0809 03:33:02.043947 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:02.043948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:02.043962 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:02.044016 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:02.045001 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.045523 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.046566 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.046789 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.046879 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.049047 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:02.049049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:02.049049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:33:02.049057 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:02.050521 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:02.051038 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:02.051040 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:02.051040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:02.051111 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:02.051371 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:02.051372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:02.051372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:02.051534 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:02.051607 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:02.051929 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 03:33:02.051996 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 03:33:02.052323 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:02.043118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:53359)
0809 03:33:02.043281 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:02.043329 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:02.043840 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:02.044097 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:02.044102 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:02.044704 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:02.044899 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.044903 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.045658 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.045659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.046473 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.046475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.046891 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.046892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.047021 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.047497 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:02.047506 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:02.048949 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:33:02.050625 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:33:02.050639 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:33:02.050642 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:33:02.050946 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:33:02.051175 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:33:02.051176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:33:02.051177 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:33:02.051309 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:33:02.051628 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:33:02.051638 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:33:02.051799 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 03:33:02.051898 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 03:33:02.052034 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
I20260809 03:33:02.052796 8308 heartbeater.cc:347] Connected to a master server at 127.27.40.190:34409
I20260809 03:33:02.052906 8308 heartbeater.cc:464] Registering TS with master...
I20260809 03:33:02.053086 8308 heartbeater.cc:511] Master 127.27.40.190:34409 requested a full tablet report, sending...
I20260809 03:33:02.053474 7647 ts_manager.cc:194] Registered new tserver with Master: 2ea483a1ca1c481da25b7e46cfbc11ed (127.27.40.131:39405)
I20260809 03:33:02.053905 7647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:53359
I20260809 03:33:02.062538 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.129.keytab.
Entry for principal oryx/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.129.keytab.
WARNING: no policy specified for oryx/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.130.keytab.
Entry for principal oryx/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.130.keytab.
WARNING: no policy specified for oryx/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.131.keytab.
Entry for principal oryx/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.131.keytab.
I20260809 03:33:02.148882 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 7742
I20260809 03:33:02.165464 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 7937
I20260809 03:33:02.176851 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 8127
I20260809 03:33:02.184244 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 7606
I20260809 03:33:02.192378 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:34409
--webserver_interface=127.27.40.190
--webserver_port=40787
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:36933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:34409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:33:02.275938 8319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:02.276186 8319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:02.276222 8319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:02.277338 8319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:33:02.277403 8319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:02.277426 8319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:02.277449 8319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:33:02.277478 8319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:33:02.278968 8319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:36933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:34409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:34409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=40787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:02.279246 8319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:02.279481 8319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:02.284492 8328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:02.284430 8327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:02.284430 8330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:02.284648 8319 server_base.cc:1034] running on GCE node
I20260809 03:33:02.284981 8319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:02.285297 8319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:02.286480 8319 hybrid_clock.cc:648] HybridClock initialized: now 1786246382286450 us; error 64 us; skew 500 ppm
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246382, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:33:02.289258 8319 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:33:02.289773 8319 webserver.cc:460] Webserver started at http://127.27.40.190:40787/ using document root <none> and password file <none>
I20260809 03:33:02.290033 8319 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:02.290143 8319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:02.291502 8319 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:33:02.292183 8340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:02.292354 8319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:33:02.292460 8319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "6f5d0838ed424e7db270114e2de7058f"
format_stamp: "Formatted at 2026-08-09 03:33:01 on dist-test-slave-gkw7"
server_key: "6fe5a7019d63ad5305f57d86f50bb8f3"
server_key_iv: "35783c830bb86b3d63d8c2e9aec01ef6"
server_key_version: "encryptionkey@0"
I20260809 03:33:02.292821 8319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:02.306216 8319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:02.308415 8319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:02.308602 8319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:02.312747 8319 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:34409
I20260809 03:33:02.312806 8410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:34409 every 8 connection(s)
I20260809 03:33:02.313166 8319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:33:02.314731 8411 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:33:02.315328 8411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f: Bootstrap starting.
I20260809 03:33:02.317673 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 8319
I20260809 03:33:02.318043 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:33845
--local_ip_for_outbound_sockets=127.27.40.129
--tserver_master_addrs=127.27.40.190:34409
--webserver_port=46295
--webserver_interface=127.27.40.129
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:36933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.40.129
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:33:02.324712 8411 log.cc:824] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:33:02.326514 8411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:33:02.326838 8411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f: Bootstrap complete.
I20260809 03:33:02.329150 8411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5d0838ed424e7db270114e2de7058f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34409 } }
I20260809 03:33:02.329404 8411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f5d0838ed424e7db270114e2de7058f, State: Initialized, Role: FOLLOWER
I20260809 03:33:02.329666 8411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5d0838ed424e7db270114e2de7058f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34409 } }
I20260809 03:33:02.329775 8411 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:33:02.329831 8411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:33:02.329880 8411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:33:02.330874 8411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5d0838ed424e7db270114e2de7058f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34409 } }
I20260809 03:33:02.331001 8411 leader_election.cc:302] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f5d0838ed424e7db270114e2de7058f; no voters:
I20260809 03:33:02.331219 8411 leader_election.cc:288] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 03:33:02.331259 8415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 03:33:02.331393 8415 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [term 2 LEADER]: Becoming Leader. State: Replica: 6f5d0838ed424e7db270114e2de7058f, State: Running, Role: LEADER
I20260809 03:33:02.331528 8411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:33:02.331561 8415 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5d0838ed424e7db270114e2de7058f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34409 } }
I20260809 03:33:02.331969 8416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6f5d0838ed424e7db270114e2de7058f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5d0838ed424e7db270114e2de7058f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34409 } } }
I20260809 03:33:02.332063 8416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [sys.catalog]: This master's current role is: LEADER
I20260809 03:33:02.332428 8418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f5d0838ed424e7db270114e2de7058f. Latest consensus state: current_term: 2 leader_uuid: "6f5d0838ed424e7db270114e2de7058f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5d0838ed424e7db270114e2de7058f" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34409 } } }
I20260809 03:33:02.332458 8426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:33:02.333179 8426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:33:02.333623 8426 catalog_manager.cc:1259] Loaded cluster ID: ef7709dd5df141db89ff14fa7172df2c
I20260809 03:33:02.333680 8426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:33:02.335083 8426 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:33:02.335304 8426 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f: Loaded TSK: 0
I20260809 03:33:02.335556 8426 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:33:02.332582 8418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f5d0838ed424e7db270114e2de7058f [sys.catalog]: This master's current role is: LEADER
W20260809 03:33:02.409293 8413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:02.409497 8413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:02.409520 8413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:02.410776 8413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:02.410821 8413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:02.410847 8413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:33:02.412286 8413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:36933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.129.keytab
--principal=oryx/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:33845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=46295
--tserver_master_addrs=127.27.40.190:34409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:02.412537 8413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:02.412792 8413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:33:02.419026 8413 server_base.cc:1034] running on GCE node
W20260809 03:33:02.418977 8451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:02.419163 8453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:02.418977 8448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:02.419513 8413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:02.419816 8413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:02.421006 8413 hybrid_clock.cc:648] HybridClock initialized: now 1786246382420986 us; error 33 us; skew 500 ppm
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246382, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:33:02.424134 8413 init.cc:377] Logged in from keytab as oryx/127.27.40.129@KRBTEST.COM (short username oryx)
I20260809 03:33:02.424675 8413 webserver.cc:460] Webserver started at http://127.27.40.129:46295/ using document root <none> and password file <none>
I20260809 03:33:02.424917 8413 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:02.425009 8413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:02.426589 8413 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:33:02.427361 8463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:02.427637 8413 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:33:02.427783 8413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "10517aad99f94fc7b8b6cdc8021a33b2"
format_stamp: "Formatted at 2026-08-09 03:33:01 on dist-test-slave-gkw7"
server_key: "e55e4e2cd3b261b55eebdbb7c7a59564"
server_key_iv: "d28889e7d9f2d3663193074fc1c49861"
server_key_version: "encryptionkey@0"
I20260809 03:33:02.428205 8413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:02.439971 8413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:02.442607 8413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:02.442790 8413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:02.443058 8413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:33:02.443394 8413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:33:02.443454 8413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:02.443503 8413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:33:02.443540 8413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:02.451759 8413 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:33845
I20260809 03:33:02.451823 8635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:33845 every 8 connection(s)
I20260809 03:33:02.452230 8413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:33:02.454908 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 8413
I20260809 03:33:02.455219 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:32887
--local_ip_for_outbound_sockets=127.27.40.130
--tserver_master_addrs=127.27.40.190:34409
--webserver_port=38343
--webserver_interface=127.27.40.130
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:36933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.40.130
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:02.462118 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:02.453007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:37373)
0809 03:33:02.453208 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:02.453255 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:02.453937 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:02.454405 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:02.454409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:02.455231 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:02.455583 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.455590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.457282 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.457283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.458177 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.458180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.458839 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.458840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.458973 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.459533 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:02.459542 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:02.461884 (+ 2342us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":101,"threads_started":1}
W20260809 03:33:02.462153 8640 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:02.453082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:37373 (local address 127.27.40.190:34409)
0809 03:33:02.453813 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:02.453821 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:02.453854 (+ 33us) server_negotiation.cc:407] Connection header received
0809 03:33:02.454163 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:02.454169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:02.454211 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:02.454326 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:02.455726 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.457155 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.458298 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.458723 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.458859 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.462009 (+ 3150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:37373: BlockingRecv error: recv got EOF from 127.27.40.129:37373 (error 108)
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":98,"threads_started":1}
W20260809 03:33:02.462666 8636 heartbeater.cc:644] Failed to heartbeat to 127.27.40.190:34409 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.40.190:34409: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:02.467226 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:02.462937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:45145)
0809 03:33:02.462981 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:02.462991 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:02.463032 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:02.463175 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:02.463176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:02.463314 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:02.463357 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.463358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.463997 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.463998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.464517 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.464518 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.464886 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.464887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.464961 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.465399 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:02.465405 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:02.467140 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 03:33:02.467279 8640 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:02.462969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:45145 (local address 127.27.40.190:34409)
0809 03:33:02.463005 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:02.463007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:02.463018 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:33:02.463083 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:02.463085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:02.463096 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:02.463172 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:02.463441 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.463908 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.464625 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.464812 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.464888 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.467214 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:45145: BlockingRecv error: recv got EOF from 127.27.40.129:45145 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:02.472050 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:02.467734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:40879)
0809 03:33:02.467776 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:02.467785 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:02.467844 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:02.467980 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:02.467981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:02.468115 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:02.468140 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.468141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.468762 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.468763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.469263 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.469265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.469731 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.469732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.469785 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.470248 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:02.470254 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:02.471980 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 03:33:02.472116 8640 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:02.467789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:40879 (local address 127.27.40.190:34409)
0809 03:33:02.467849 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:02.467852 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:02.467878 (+ 26us) server_negotiation.cc:407] Connection header received
0809 03:33:02.467898 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:02.467899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:02.467908 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:02.467976 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:02.468215 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.468671 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.469388 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.469648 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.469729 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.472067 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:40879: BlockingRecv error: recv got EOF from 127.27.40.129:40879 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 03:33:02.472268 8636 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:33:02.543606 8641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:02.543808 8641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:02.543830 8641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:02.544924 8641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:02.544960 8641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:02.544986 8641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:33:02.546785 8641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:36933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.130.keytab
--principal=oryx/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:32887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=38343
--tserver_master_addrs=127.27.40.190:34409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:02.547030 8641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:02.547268 8641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:33:02.552520 8641 server_base.cc:1034] running on GCE node
W20260809 03:33:02.552485 8649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:02.552486 8648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:02.552479 8651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:02.552932 8641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:02.553227 8641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:02.554388 8641 hybrid_clock.cc:648] HybridClock initialized: now 1786246382554371 us; error 31 us; skew 500 ppm
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246382, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:33:02.556988 8641 init.cc:377] Logged in from keytab as oryx/127.27.40.130@KRBTEST.COM (short username oryx)
I20260809 03:33:02.557389 8641 webserver.cc:460] Webserver started at http://127.27.40.130:38343/ using document root <none> and password file <none>
I20260809 03:33:02.557648 8641 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:02.557715 8641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:02.559015 8641 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:33:02.559675 8659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:02.559882 8641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:33:02.559953 8641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "eb049bed7df042b3874f6c3082dcb459"
format_stamp: "Formatted at 2026-08-09 03:33:01 on dist-test-slave-gkw7"
server_key: "472c07f40386503a853627a28368f979"
server_key_iv: "c54876aa1a06058eea721c302f41b38f"
server_key_version: "encryptionkey@0"
I20260809 03:33:02.560303 8641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:02.570477 8641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:02.572755 8641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:02.572877 8641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:02.573062 8641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:33:02.573390 8641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:33:02.573426 8641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:02.573448 8641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:33:02.573495 8641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:02.579524 8641 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:32887
I20260809 03:33:02.579566 8819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:32887 every 8 connection(s)
I20260809 03:33:02.579919 8641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:33:02.581833 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 8641
I20260809 03:33:02.582199 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:39405
--local_ip_for_outbound_sockets=127.27.40.131
--tserver_master_addrs=127.27.40.190:34409
--webserver_port=45357
--webserver_interface=127.27.40.131
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:36933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.40.131
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 03:33:02.588508 8640 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:02.580675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:35019 (local address 127.27.40.190:34409)
0809 03:33:02.580723 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:02.580725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:02.580943 (+ 218us) server_negotiation.cc:407] Connection header received
0809 03:33:02.581438 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:02.581439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:02.581453 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:02.581520 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:02.582943 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.583467 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.584690 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.585005 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.585091 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.588449 (+ 3358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:35019: BlockingRecv error: recv got EOF from 127.27.40.130:35019 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 03:33:02.588636 8822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:02.580639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:35019)
0809 03:33:02.580807 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:02.580850 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:02.581355 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:02.581635 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:02.581639 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:02.582535 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:02.582826 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.582833 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.583562 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.583563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.584601 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.584604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.585107 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.585108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.585234 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.585824 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:02.585833 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:02.588369 (+ 2536us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
W20260809 03:33:02.588874 8820 heartbeater.cc:644] Failed to heartbeat to 127.27.40.190:34409 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.40.190:34409: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:02.594287 8822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:02.589136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:46505)
0809 03:33:02.589188 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:02.589199 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:02.589238 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:02.589382 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:02.589383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:02.589597 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:02.589657 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.589659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.590285 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.590286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.591185 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.591188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.591575 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.591576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.591656 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.592097 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:02.592103 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:02.594203 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 03:33:02.594342 8640 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:02.589190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:46505 (local address 127.27.40.190:34409)
0809 03:33:02.589228 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:02.589229 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:02.589238 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:33:02.589294 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:02.589295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:02.589302 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:02.589351 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:02.589747 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.590205 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.591296 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.591482 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.591550 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.594278 (+ 2728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:46505: BlockingRecv error: recv got EOF from 127.27.40.130:46505 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:02.599328 8824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:02.594555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:49603)
0809 03:33:02.594756 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:02.594767 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:02.594848 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:02.594988 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:02.594989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:02.595152 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:02.595194 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.595195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.595932 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.595933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.596491 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.596492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.597017 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.597019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.597076 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.597562 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:02.597567 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:02.599243 (+ 1676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":151,"threads_started":1}
W20260809 03:33:02.599382 8640 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:02.594602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:49603 (local address 127.27.40.190:34409)
0809 03:33:02.594639 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:02.594641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:02.594836 (+ 195us) server_negotiation.cc:407] Connection header received
0809 03:33:02.594906 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:02.594907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:02.594916 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:02.594984 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:02.595293 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.595856 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.596584 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.596909 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.596994 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.599325 (+ 2331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:49603: BlockingRecv error: recv got EOF from 127.27.40.130:49603 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 03:33:02.599447 8820 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:33:02.668385 8823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:02.668586 8823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:02.668609 8823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:02.669842 8823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:02.669883 8823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:02.669909 8823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:33:02.671361 8823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:36933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/krb5kdc/oryx_127.27.40.131.keytab
--principal=oryx/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:39405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=45357
--tserver_master_addrs=127.27.40.190:34409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:02.671622 8823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:02.671854 8823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:02.676956 8833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:02.676928 8837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:02.677042 8834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:02.677193 8823 server_base.cc:1034] running on GCE node
I20260809 03:33:02.677431 8823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:02.677701 8823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:02.678884 8823 hybrid_clock.cc:648] HybridClock initialized: now 1786246382678862 us; error 39 us; skew 500 ppm
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246382, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:33:02.681418 8823 init.cc:377] Logged in from keytab as oryx/127.27.40.131@KRBTEST.COM (short username oryx)
I20260809 03:33:02.681941 8823 webserver.cc:460] Webserver started at http://127.27.40.131:45357/ using document root <none> and password file <none>
I20260809 03:33:02.682197 8823 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:02.682283 8823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:02.683621 8823 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:33:02.684185 8845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:02.684358 8823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:33:02.684461 8823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "2ea483a1ca1c481da25b7e46cfbc11ed"
format_stamp: "Formatted at 2026-08-09 03:33:02 on dist-test-slave-gkw7"
server_key: "4b0405112935c5e6c2ddf8d1e8ed59b1"
server_key_iv: "23387d9df3d82f0962b557ba505f865d"
server_key_version: "encryptionkey@0"
I20260809 03:33:02.684791 8823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:02.706714 8823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:02.708716 8823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:02.708885 8823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:02.709132 8823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:33:02.709457 8823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:33:02.709514 8823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:02.709590 8823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:33:02.709640 8823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:02.715547 8823 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:39405
I20260809 03:33:02.715606 9003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:39405 every 8 connection(s)
I20260809 03:33:02.715940 8823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:33:02.718567 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 8823
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 03:33:02.724933 8640 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:02.716673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:45703 (local address 127.27.40.190:34409)
0809 03:33:02.716763 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:02.716765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:02.716914 (+ 149us) server_negotiation.cc:407] Connection header received
0809 03:33:02.717598 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:02.717599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:02.717610 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:02.717659 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:02.718856 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.719616 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.721012 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.721313 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.721392 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.724884 (+ 3492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:45703: BlockingRecv error: recv got EOF from 127.27.40.131:45703 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:33:02.725016 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:02.716624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:45703)
0809 03:33:02.716797 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:02.716833 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:02.717455 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:02.717747 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:02.717750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:02.718426 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:02.718749 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.718757 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.719716 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.719717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.720901 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.720904 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.721404 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.721404 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.721504 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.722101 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:02.722109 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:02.724816 (+ 2707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
W20260809 03:33:02.725243 9005 heartbeater.cc:644] Failed to heartbeat to 127.27.40.190:34409 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.40.190:34409: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
I20260809 03:33:02.726976 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:02.719140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39762 (local address 127.27.40.190:34409)
0809 03:33:02.719275 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:02.719277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:02.719286 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:33:02.719307 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:02.719308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:02.719324 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:02.719379 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:02.720044 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.720783 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.721870 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.722061 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.722181 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.723462 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:02.723466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:02.723466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:33:02.723478 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:02.725450 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:02.725895 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:02.725897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:02.725898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:02.726023 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:02.726223 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:02.726223 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:02.726224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:02.726508 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:02.726664 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:02.726761 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 03:33:02.726858 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:02.730135 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:02.725467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:36933)
0809 03:33:02.725513 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:02.725523 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:02.725590 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:02.725724 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:02.725725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:02.725869 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:02.725918 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.725919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.726760 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.726761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.727506 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.727508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.727959 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.727960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.728032 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.728405 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:02.728410 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:02.730082 (+ 1672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 03:33:02.730260 8640 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:02.725515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:36933 (local address 127.27.40.190:34409)
0809 03:33:02.725579 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:02.725581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:02.725596 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:33:02.725646 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:02.725647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:02.725653 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:02.725698 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:02.725994 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.726682 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.727602 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.727883 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.727937 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.730188 (+ 2251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:36933: BlockingRecv error: recv got EOF from 127.27.40.131:36933 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:02 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:02.734730 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:02.730407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:33677)
0809 03:33:02.730456 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:02.730466 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:02.730502 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:02.730704 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:02.730705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:02.730823 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:02.730848 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.730849 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.731572 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.731573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.732097 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:02.732099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.732461 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.732461 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:02.732497 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.732947 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:02.732952 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:02.734650 (+ 1698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 03:33:02.734774 8640 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:02.730509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:33677 (local address 127.27.40.190:34409)
0809 03:33:02.730557 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:02.730558 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:02.730572 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:33:02.730590 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:02.730590 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:02.730597 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:02.730683 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:02.730933 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.731433 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.732166 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:02.732396 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:02.732450 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:02.734717 (+ 2267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:33677: BlockingRecv error: recv got EOF from 127.27.40.131:33677 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 03:33:02.734900 9005 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 03:33:03 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:03 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:03.477617 9013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:03.472748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:49517)
0809 03:33:03.472930 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:03.472944 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:03.473005 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:03.473162 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:03.473163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:03.473338 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:03.473390 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:03.473392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:03.474228 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:03.474229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:03.474820 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:03.474822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:03.475214 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:03.475216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:03.475261 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:03.475714 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:03.475718 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:03.477487 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":99,"threads_started":1}
W20260809 03:33:03.477717 9014 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:03.472863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:49517 (local address 127.27.40.190:34409)
0809 03:33:03.473011 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:03.473014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:03.473026 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:33:03.473059 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:03.473060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:03.473074 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:03.473148 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:03.473487 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:03.474142 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:03.474918 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:03.475140 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:03.475204 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:03.477644 (+ 2440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:49517: BlockingRecv error: recv got EOF from 127.27.40.129:49517 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
Aug 09 03:33:03 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:03 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:03.605008 9015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:03.599859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:45543)
0809 03:33:03.600019 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:03.600030 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:03.600097 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:03.600226 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:03.600227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:03.600411 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:03.600465 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:03.600466 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:03.601197 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:03.601198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:03.601874 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:03.601877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:03.602384 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:03.602386 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:03.602504 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:03.602967 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:03.602972 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:03.604938 (+ 1966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
W20260809 03:33:03.605135 9014 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:03.599957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:45543 (local address 127.27.40.190:34409)
0809 03:33:03.600014 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:03.600016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:03.600100 (+ 84us) server_negotiation.cc:407] Connection header received
0809 03:33:03.600142 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:03.600142 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:03.600153 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:03.600204 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:03.600561 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:03.601111 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:03.601992 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:03.602299 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:03.602365 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:03.605047 (+ 2682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:45543: BlockingRecv error: recv got EOF from 127.27.40.130:45543 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 03:33:03 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:03 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:03.740296 9017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:03.735329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:50617)
0809 03:33:03.735487 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:03.735497 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:03.735556 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:03.735705 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:03.735706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:03.735879 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:03.735934 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:03.735936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:03.736595 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:03.736596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:03.737261 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:03.737263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:03.737741 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:03.737742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:03.737807 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:03.738319 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:03.738327 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:03.740222 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
W20260809 03:33:03.740398 9014 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:03.735387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:50617 (local address 127.27.40.190:34409)
0809 03:33:03.735464 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:03.735471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:03.735555 (+ 84us) server_negotiation.cc:407] Connection header received
0809 03:33:03.735620 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:03.735621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:03.735634 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:03.735688 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:03.735994 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:03.736505 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:03.737343 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:03.737628 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:03.737702 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:03.740329 (+ 2627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:50617: BlockingRecv error: recv got EOF from 127.27.40.131:50617 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 03:33:04 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:04 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:04.483191 9018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:04.478147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:41185)
0809 03:33:04.478302 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:04.478314 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:04.478366 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:04.478484 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:04.478485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:04.478679 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:04.478732 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:04.478734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:04.479437 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:04.479438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:04.479993 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:04.479994 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:04.480362 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:04.480363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:04.480410 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:04.480937 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:04.480942 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:04.483068 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
W20260809 03:33:04.483207 9020 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:04.478200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:41185 (local address 127.27.40.190:34409)
0809 03:33:04.478336 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:04.478342 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:04.478371 (+ 29us) server_negotiation.cc:407] Connection header received
0809 03:33:04.478404 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:04.478404 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:04.478416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:04.478469 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:04.478800 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:04.479362 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:04.480064 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:04.480295 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:04.480350 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:04.483145 (+ 2795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:41185: BlockingRecv error: recv got EOF from 127.27.40.129:41185 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
Aug 09 03:33:04 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:04 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:04.610339 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:04.605591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:52543)
0809 03:33:04.605761 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:04.605780 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:04.605836 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:04.605979 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:04.605980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:04.606151 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:04.606205 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:04.606206 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:04.606878 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:04.606879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:04.607405 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:04.607407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:04.607761 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:04.607762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:04.607803 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:04.608329 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:04.608335 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:04.610272 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":96,"threads_started":1}
W20260809 03:33:04.610551 9020 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:04.605608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:52543 (local address 127.27.40.190:34409)
0809 03:33:04.605681 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:04.605684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:04.605858 (+ 174us) server_negotiation.cc:407] Connection header received
0809 03:33:04.605889 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:04.605890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:04.605903 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:04.605963 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:04.606293 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:04.606809 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:04.607486 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:04.607692 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:04.607750 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:04.610485 (+ 2735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:52543: BlockingRecv error: recv got EOF from 127.27.40.130:52543 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 03:33:04 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:04 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:04.745836 9022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:04.740802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:36299)
0809 03:33:04.741003 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:04.741017 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:04.741088 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:04.741225 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:04.741226 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:04.741396 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:04.741449 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:04.741450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:04.742174 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:04.742175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:04.742723 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:04.742725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:04.743061 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:04.743062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:04.743103 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:04.743706 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:04.743711 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:04.745740 (+ 2029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":81,"threads_started":1}
W20260809 03:33:04.745913 9020 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:04.740990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:36299 (local address 127.27.40.190:34409)
0809 03:33:04.741060 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:04.741062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:04.741072 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:33:04.741136 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:04.741138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:04.741150 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:04.741201 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:04.741514 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:04.742080 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:04.742808 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:04.742994 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:04.743048 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:04.745864 (+ 2816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:36299: BlockingRecv error: recv got EOF from 127.27.40.131:36299 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 03:33:05 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:05 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:05.494189 9024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:05.483709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:46199)
0809 03:33:05.483883 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:05.483897 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:05.483982 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:05.484144 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:05.484146 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:05.484314 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:05.484365 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:05.484366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:05.485032 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:05.485033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:05.485694 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:05.485696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:05.486039 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:05.486040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:05.486083 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:05.492156 (+ 6073us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:05.492162 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:05.494116 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":96,"threads_started":1}
W20260809 03:33:05.494318 9026 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:05.483818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:46199 (local address 127.27.40.190:34409)
0809 03:33:05.483944 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:05.483946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:05.483956 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:33:05.484037 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:05.484038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:05.484048 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:05.484105 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:05.484437 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:05.484953 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:05.485771 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:05.485970 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:05.486027 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:05.494228 (+ 8201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:46199: BlockingRecv error: recv got EOF from 127.27.40.129:46199 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
Aug 09 03:33:05 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:05 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:05.615443 9027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:05.610882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:32889)
0809 03:33:05.611028 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:05.611040 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:05.611109 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:05.611250 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:05.611251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:05.611423 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:05.611477 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:05.611478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:05.612167 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:05.612169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:05.612717 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:05.612719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:05.613054 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:05.613054 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:05.613097 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:05.613622 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:05.613633 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:05.615384 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
W20260809 03:33:05.615624 9026 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:05.610953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:32889 (local address 127.27.40.190:34409)
0809 03:33:05.611028 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:05.611030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:05.611099 (+ 69us) server_negotiation.cc:407] Connection header received
0809 03:33:05.611181 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:05.611182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:05.611193 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:05.611244 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:05.611552 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:05.612059 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:05.612802 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:05.612988 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:05.613044 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:05.615542 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:32889: BlockingRecv error: recv got EOF from 127.27.40.130:32889 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 03:33:05 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:05 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:05.751504 9031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:05.746453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:55483)
0809 03:33:05.746634 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:05.746645 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:05.746716 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:05.746858 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:05.746859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:05.747031 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:05.747085 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:05.747087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:05.747741 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:05.747742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:05.748282 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:05.748284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:05.748748 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:05.748749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:05.748818 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:05.749459 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:05.749463 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:05.751444 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":65,"threads_started":1}
W20260809 03:33:05.751653 9026 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:05.746573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:55483 (local address 127.27.40.190:34409)
0809 03:33:05.746636 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:05.746638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:05.746697 (+ 59us) server_negotiation.cc:407] Connection header received
0809 03:33:05.746762 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:05.746763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:05.746778 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:05.746828 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:05.747166 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:05.747670 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:05.748406 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:05.748656 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:05.748746 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:05.751553 (+ 2807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:55483: BlockingRecv error: recv got EOF from 127.27.40.131:55483 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 09 03:33:06 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:06 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:06.499863 9032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:06.494738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:40777)
0809 03:33:06.494895 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:06.494906 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:06.495007 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:06.495162 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:06.495163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:06.495339 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:06.495393 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:06.495394 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:06.496128 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:06.496129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:06.496682 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:06.496684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:06.497017 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:06.497018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:06.497067 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:06.497619 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:06.497624 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:06.499803 (+ 2179us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
W20260809 03:33:06.500011 9033 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:06.494824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:40777 (local address 127.27.40.190:34409)
0809 03:33:06.494965 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:06.494967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:06.494992 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:33:06.495074 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:06.495075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:06.495086 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:06.495146 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:06.495460 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:06.496002 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:06.496750 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:06.496942 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:06.496996 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:06.499915 (+ 2919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:40777: BlockingRecv error: recv got EOF from 127.27.40.129:40777 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
Aug 09 03:33:06 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:06 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:06.621413 9034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:06.615947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:53375)
0809 03:33:06.616110 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:06.616123 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:06.616198 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:06.616349 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:06.616350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:06.616527 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:06.616582 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:06.616583 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:06.617270 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:06.617271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:06.618148 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:06.618151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:06.618571 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:06.618572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:06.618625 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:06.619220 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:06.619225 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:06.621353 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
W20260809 03:33:06.621577 9033 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:06.616056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:53375 (local address 127.27.40.190:34409)
0809 03:33:06.616135 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:06.616137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:06.616173 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:33:06.616269 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:06.616270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:06.616281 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:06.616325 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:06.616667 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:06.617193 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:06.618257 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:06.618458 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:06.618519 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:06.621479 (+ 2960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:53375: BlockingRecv error: recv got EOF from 127.27.40.130:53375 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 03:33:06 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:06 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:06.756786 9036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:06.752083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:53235)
0809 03:33:06.752249 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:06.752261 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:06.752327 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:06.752509 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:06.752510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:06.752694 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:06.752747 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:06.752749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:06.753472 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:06.753473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:06.754060 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:06.754062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:06.754410 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:06.754411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:06.754454 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:06.754921 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:06.754925 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:06.756727 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":104,"threads_started":1}
W20260809 03:33:06.756923 9033 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:06.752144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:53235 (local address 127.27.40.190:34409)
0809 03:33:06.752228 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:06.752230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:06.752306 (+ 76us) server_negotiation.cc:407] Connection header received
0809 03:33:06.752413 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:06.752414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:06.752430 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:06.752490 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:06.752826 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:06.753369 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:06.754147 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:06.754338 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:06.754401 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:06.756833 (+ 2432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:53235: BlockingRecv error: recv got EOF from 127.27.40.131:53235 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 03:33:07 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:07 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:07.505856 9038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:07.500403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:56891)
0809 03:33:07.500597 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:07.500609 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:07.500672 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:07.500809 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:07.500810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:07.500980 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:07.501033 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:07.501034 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:07.501792 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:07.501794 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:07.502610 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:07.502632 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:07.503240 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:07.503241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:07.503294 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:07.503793 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:07.503797 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:07.505793 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":102,"threads_started":1}
W20260809 03:33:07.506043 9041 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:07.500513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:56891 (local address 127.27.40.190:34409)
0809 03:33:07.500656 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:07.500659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:07.500676 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:33:07.500720 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:07.500720 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:07.500731 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:07.500792 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:07.501118 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:07.501708 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:07.502749 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:07.503023 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:07.503140 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:07.505920 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:56891: BlockingRecv error: recv got EOF from 127.27.40.129:56891 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
Aug 09 03:33:07 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:07 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:07.626866 9042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:07.621884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:51511)
0809 03:33:07.622020 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:07.622035 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:07.622137 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:07.622267 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:07.622268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:07.622455 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:07.622512 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:07.622513 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:07.623178 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:07.623178 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:07.623747 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:07.623749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:07.624084 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:07.624086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:07.624131 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:07.624643 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:07.624648 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:07.626756 (+ 2108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
W20260809 03:33:07.626920 9041 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:07.621993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:51511 (local address 127.27.40.190:34409)
0809 03:33:07.622042 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:07.622045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:07.622160 (+ 115us) server_negotiation.cc:407] Connection header received
0809 03:33:07.622202 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:07.622203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:07.622213 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:07.622260 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:07.622592 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:07.623090 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:07.623824 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:07.624010 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:07.624061 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:07.626849 (+ 2788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:51511: BlockingRecv error: recv got EOF from 127.27.40.130:51511 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 03:33:07 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:07 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:07.762305 9043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:07.757356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:38005)
0809 03:33:07.757600 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:07.757613 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:07.757705 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:07.757868 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:07.757869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:07.758062 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:07.758114 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:07.758116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:07.758808 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:07.758809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:07.759407 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:07.759409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:07.759797 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:07.759798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:07.759866 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:07.760333 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:07.760337 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:07.762180 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":114,"threads_started":1}
W20260809 03:33:07.762354 9041 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:07.757459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:38005 (local address 127.27.40.190:34409)
0809 03:33:07.757573 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:07.757576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:07.757686 (+ 110us) server_negotiation.cc:407] Connection header received
0809 03:33:07.757766 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:07.757767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:07.757782 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:07.757860 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:07.758197 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:07.758718 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:07.759499 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:07.759705 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:07.759793 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:07.762286 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:38005: BlockingRecv error: recv got EOF from 127.27.40.131:38005 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 03:33:08 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:08 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:08.511049 9044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:08.506371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:57379)
0809 03:33:08.506533 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:08.506545 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:08.506606 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:08.506749 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:08.506749 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:08.506919 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:08.506972 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:08.506973 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:08.507618 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:08.507619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:08.508180 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:08.508182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:08.508512 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:08.508513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:08.508567 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:08.509057 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:08.509062 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:08.510951 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":102,"threads_started":1}
W20260809 03:33:08.511127 9045 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:08.506457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:57379 (local address 127.27.40.190:34409)
0809 03:33:08.506593 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:08.506595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:08.506604 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:33:08.506653 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:08.506654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:08.506664 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:08.506728 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:08.507041 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:08.507549 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:08.508258 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:08.508447 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:08.508496 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:08.511050 (+ 2554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:57379: BlockingRecv error: recv got EOF from 127.27.40.129:57379 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
Aug 09 03:33:08 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:08 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:08.632607 9046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:08.627373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:46705)
0809 03:33:08.627672 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:08.627683 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:08.627741 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:08.627916 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:08.627917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:08.628088 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:08.628142 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:08.628144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:08.628805 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:08.628805 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:08.629361 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:08.629363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:08.629911 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:08.629913 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:08.629965 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:08.630505 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:08.630510 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:08.632547 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":216,"threads_started":1}
W20260809 03:33:08.632721 9045 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:08.627567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:46705 (local address 127.27.40.190:34409)
0809 03:33:08.627641 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:08.627643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:08.627765 (+ 122us) server_negotiation.cc:407] Connection header received
0809 03:33:08.627812 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:08.627813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:08.627831 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:08.627896 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:08.628216 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:08.628732 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:08.629443 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:08.629814 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:08.629879 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:08.632651 (+ 2772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:46705: BlockingRecv error: recv got EOF from 127.27.40.130:46705 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 03:33:08 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:08 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:08.767704 9048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:08.762880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:41359)
0809 03:33:08.763055 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:08.763068 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:08.763150 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:08.763330 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:08.763330 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:08.763513 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:08.763568 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:08.763570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:08.764258 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:08.764259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:08.764816 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:08.764818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:08.765220 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:08.765221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:08.765262 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:08.765740 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:08.765745 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:08.767615 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":82,"threads_started":1}
W20260809 03:33:08.767789 9045 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:08.762950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:41359 (local address 127.27.40.190:34409)
0809 03:33:08.763063 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:08.763066 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:08.763153 (+ 87us) server_negotiation.cc:407] Connection header received
0809 03:33:08.763244 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:08.763245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:08.763259 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:08.763309 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:08.763635 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:08.764161 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:08.764894 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:08.765082 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:08.765211 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:08.767722 (+ 2511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:41359: BlockingRecv error: recv got EOF from 127.27.40.131:41359 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 09 03:33:09 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:09 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:09.516633 9050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:09.511582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:57689)
0809 03:33:09.511733 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:09.511748 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:09.511811 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:09.511964 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:09.511965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:09.512169 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:09.512224 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:09.512226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:09.512898 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:09.512899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:09.513449 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:09.513451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:09.513878 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:09.513879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:09.513934 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:09.514485 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:09.514497 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:09.516571 (+ 2074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
W20260809 03:33:09.516805 9051 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:09.511671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:57689 (local address 127.27.40.190:34409)
0809 03:33:09.511847 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:09.511850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:09.511858 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:33:09.511887 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:09.511887 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:09.511898 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:09.511952 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:09.512302 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:09.512821 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:09.513562 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:09.513778 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:09.513843 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:09.516715 (+ 2872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:57689: BlockingRecv error: recv got EOF from 127.27.40.129:57689 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
Aug 09 03:33:09 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:09 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:09.639609 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:09.633179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:45577)
0809 03:33:09.633371 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:09.633383 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:09.633479 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:09.633669 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:09.633670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:09.633863 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:09.633915 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:09.633917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:09.634797 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:09.634798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:09.635710 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:09.635714 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:09.636108 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:09.636110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:09.636200 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:09.636834 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:09.636840 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:09.639500 (+ 2660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":105,"threads_started":1}
W20260809 03:33:09.639670 9051 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:09.633267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:45577 (local address 127.27.40.190:34409)
0809 03:33:09.633354 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:09.633356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:09.633461 (+ 105us) server_negotiation.cc:407] Connection header received
0809 03:33:09.633562 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:09.633563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:09.633577 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:09.633652 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:09.633999 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:09.634697 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:09.635808 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:09.636008 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:09.636071 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:09.639604 (+ 3533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:45577: BlockingRecv error: recv got EOF from 127.27.40.130:45577 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 03:33:09 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:09 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:09.773013 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:09.768272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:56035)
0809 03:33:09.768416 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:09.768426 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:09.768494 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:09.768622 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:09.768622 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:09.768804 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:09.768858 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:09.768859 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:09.769676 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:09.769676 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:09.770208 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:09.770210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:09.770549 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:09.770550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:09.770591 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:09.771045 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:09.771050 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:09.772911 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
W20260809 03:33:09.773077 9051 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:09.768323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:56035 (local address 127.27.40.190:34409)
0809 03:33:09.768398 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:09.768400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:09.768506 (+ 106us) server_negotiation.cc:407] Connection header received
0809 03:33:09.768534 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:09.768535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:09.768547 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:09.768609 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:09.768963 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:09.769595 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:09.770292 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:09.770480 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:09.770537 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:09.772997 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:56035: BlockingRecv error: recv got EOF from 127.27.40.131:56035 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 03:33:10 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:10 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:10.526607 9054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:10.517207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:46737)
0809 03:33:10.517396 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:10.517408 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:10.517470 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:10.517710 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:10.517711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:10.517890 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:10.517949 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:10.517951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:10.518627 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:10.518628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:10.519154 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:10.519155 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:10.519492 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:10.519493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:10.519535 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:10.524591 (+ 5056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:10.524612 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:10.526532 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":134,"threads_started":1}
W20260809 03:33:10.526764 9056 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:10.517389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:46737 (local address 127.27.40.190:34409)
0809 03:33:10.517525 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:10.517568 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:10.517577 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:33:10.517612 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:10.517613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:10.517623 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:10.517689 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:10.518025 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:10.518551 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:10.519231 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:10.519423 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:10.519478 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:10.526664 (+ 7186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:46737: BlockingRecv error: recv got EOF from 127.27.40.129:46737 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
Aug 09 03:33:10 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:10 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:10.645298 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:10.640224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:43491)
0809 03:33:10.640443 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:10.640454 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:10.640555 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:10.640725 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:10.640726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:10.640919 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:10.640968 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:10.640970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:10.641917 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:10.641919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:10.642462 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:10.642464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:10.642845 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:10.642846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:10.642913 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:10.643393 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:10.643398 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:10.645234 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":94,"threads_started":1}
W20260809 03:33:10.645442 9056 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:10.640335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:43491 (local address 127.27.40.190:34409)
0809 03:33:10.640424 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:10.640426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:10.640535 (+ 109us) server_negotiation.cc:407] Connection header received
0809 03:33:10.640621 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:10.640622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:10.640644 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:10.640701 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:10.641057 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:10.641833 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:10.642549 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:10.642757 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:10.642834 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:10.645353 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:43491: BlockingRecv error: recv got EOF from 127.27.40.130:43491 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 03:33:10 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:10 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:10.778244 9058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:10.773582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:58343)
0809 03:33:10.773754 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:10.773767 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:10.773831 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:10.773947 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:10.773947 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:10.774143 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:10.774196 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:10.774198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:10.774830 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:10.774831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:10.775371 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:10.775372 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:10.775712 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:10.775713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:10.775753 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:10.776357 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:10.776362 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:10.778184 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":112,"threads_started":1}
W20260809 03:33:10.778378 9056 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:10.773658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:58343 (local address 127.27.40.190:34409)
0809 03:33:10.773722 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:10.773724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:10.773830 (+ 106us) server_negotiation.cc:407] Connection header received
0809 03:33:10.773886 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:10.773887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:10.773899 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:10.773940 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:10.774258 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:10.774759 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:10.775448 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:10.775643 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:10.775697 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:10.778293 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:58343: BlockingRecv error: recv got EOF from 127.27.40.131:58343 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 03:33:11 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:11 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:11.532497 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:11.527095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:37509)
0809 03:33:11.527263 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:11.527276 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:11.527337 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:11.527585 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:11.527587 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:11.527794 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:11.527856 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:11.527858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:11.528654 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:11.528655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:11.529195 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:11.529197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:11.529628 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:11.529629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:11.529690 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:11.530214 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:11.530219 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:11.532436 (+ 2217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":104,"threads_started":1}
W20260809 03:33:11.532711 9060 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:11.527225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:37509 (local address 127.27.40.190:34409)
0809 03:33:11.527377 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:11.527379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:11.527387 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:33:11.527416 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:11.527416 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:11.527427 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:11.527498 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:11.527960 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:11.528552 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:11.529282 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:11.529495 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:11.529566 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:11.532606 (+ 3040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:37509: BlockingRecv error: recv got EOF from 127.27.40.129:37509 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
Aug 09 03:33:11 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:11 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:11.650722 9061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:11.645783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:49617)
0809 03:33:11.645925 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:11.645940 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:11.646014 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:11.646174 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:11.646175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:11.646367 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:11.646418 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:11.646419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:11.647104 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:11.647106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:11.647669 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:11.647671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:11.648033 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:11.648034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:11.648093 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:11.648606 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:11.648611 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:11.650613 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
W20260809 03:33:11.650763 9060 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:11.645898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:49617 (local address 127.27.40.190:34409)
0809 03:33:11.645947 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:11.645949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:11.646009 (+ 60us) server_negotiation.cc:407] Connection header received
0809 03:33:11.646070 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:11.646071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:11.646086 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:11.646165 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:11.646501 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:11.647017 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:11.647760 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:11.647958 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:11.648034 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:11.650697 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:49617: BlockingRecv error: recv got EOF from 127.27.40.130:49617 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 03:33:11 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:11 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:11.783435 9063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:11.778782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:52669)
0809 03:33:11.778928 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:11.778940 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:11.779019 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:11.779159 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:11.779160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:11.779341 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:11.779397 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:11.779398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:11.780058 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:11.780060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:11.780599 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:11.780601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:11.780943 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:11.780945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:11.781000 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:11.781444 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:11.781448 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:11.783374 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":73,"threads_started":1}
W20260809 03:33:11.783617 9060 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:11.778831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:52669 (local address 127.27.40.190:34409)
0809 03:33:11.778885 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:11.778888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:11.779019 (+ 131us) server_negotiation.cc:407] Connection header received
0809 03:33:11.779085 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:11.779086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:11.779098 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:11.779156 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:11.779460 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:11.779972 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:11.780674 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:11.780850 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:11.780902 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:11.783525 (+ 2623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:52669: BlockingRecv error: recv got EOF from 127.27.40.131:52669 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 03:33:12 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:12 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:12.538173 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:12.533000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:55401)
0809 03:33:12.533216 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:12.533228 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:12.533307 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:12.533446 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:12.533447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:12.533673 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:12.533745 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:12.533747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:12.534422 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:12.534423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:12.534958 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:12.534960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:12.535352 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:12.535354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:12.535410 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:12.535853 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:12.535859 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:12.538079 (+ 2220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":141,"threads_started":1}
W20260809 03:33:12.538270 9067 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:12.533089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:55401 (local address 127.27.40.190:34409)
0809 03:33:12.533233 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:12.533236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:12.533311 (+ 75us) server_negotiation.cc:407] Connection header received
0809 03:33:12.533357 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:12.533358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:12.533369 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:12.533429 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:12.533812 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:12.534349 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:12.535031 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:12.535250 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:12.535312 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:12.538196 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:55401: BlockingRecv error: recv got EOF from 127.27.40.129:55401 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
Aug 09 03:33:12 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:12 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:12.655822 9068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:12.651322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:38853)
0809 03:33:12.651465 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:12.651485 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:12.651550 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:12.651671 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:12.651671 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:12.651845 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:12.651899 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:12.651900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:12.652538 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:12.652539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:12.653058 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:12.653060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:12.653480 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:12.653480 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:12.653523 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:12.653966 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:12.653971 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:12.655761 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":61,"threads_started":1}
W20260809 03:33:12.655946 9067 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:12.651429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:38853 (local address 127.27.40.190:34409)
0809 03:33:12.651480 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:12.651482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:12.651552 (+ 70us) server_negotiation.cc:407] Connection header received
0809 03:33:12.651603 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:12.651604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:12.651616 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:12.651659 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:12.651960 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:12.652467 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:12.653152 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:12.653402 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:12.653465 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:12.655863 (+ 2398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:38853: BlockingRecv error: recv got EOF from 127.27.40.130:38853 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 03:33:12 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:12 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:12.788981 9069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:12.783923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:51711)
0809 03:33:12.784102 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:12.784122 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:12.784179 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:12.784453 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:12.784454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:12.784645 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:12.784699 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:12.784700 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:12.785376 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:12.785377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:12.785946 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:12.785948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:12.786288 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:12.786289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:12.786332 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:12.786792 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:12.786797 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:12.788905 (+ 2108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":103,"threads_started":1}
W20260809 03:33:12.789089 9067 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:12.784149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:51711 (local address 127.27.40.190:34409)
0809 03:33:12.784287 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:12.784289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:12.784302 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:33:12.784339 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:12.784340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:12.784356 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:12.784424 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:12.784772 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:12.785296 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:12.786015 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:12.786211 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:12.786275 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:12.788994 (+ 2719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:51711: BlockingRecv error: recv got EOF from 127.27.40.131:51711 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 09 03:33:13 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:13 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:13.543967 9070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:13.538799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:41807)
0809 03:33:13.538988 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:13.539000 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:13.539055 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:13.539225 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:13.539225 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:13.539462 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:13.539525 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:13.539527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:13.540306 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:13.540308 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:13.540965 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:13.540968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:13.541348 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:13.541350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:13.541416 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:13.541913 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:13.541918 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:13.543856 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":98,"threads_started":1}
W20260809 03:33:13.544018 9071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:13.538880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:41807 (local address 127.27.40.190:34409)
0809 03:33:13.539047 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:13.539049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:13.539060 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:33:13.539119 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:13.539120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:13.539132 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:13.539200 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:13.539595 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:13.540214 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:13.541057 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:13.541267 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:13.541347 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:13.543942 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:41807: BlockingRecv error: recv got EOF from 127.27.40.129:41807 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
Aug 09 03:33:13 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:13 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:13.661674 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:13.656410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:43179)
0809 03:33:13.656556 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:13.656566 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:13.656621 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:13.656763 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:13.656764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:13.656952 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:13.657004 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:13.657005 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:13.657725 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:13.657726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:13.658475 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:13.658479 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:13.658850 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:13.658850 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:13.658892 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:13.659342 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:13.659347 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:13.661584 (+ 2237us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
W20260809 03:33:13.661722 9071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:13.656489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:43179 (local address 127.27.40.190:34409)
0809 03:33:13.656599 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:13.656601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:13.656613 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:33:13.656672 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:13.656674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:13.656686 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:13.656754 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:13.657077 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:13.657654 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:13.658575 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:13.658780 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:13.658837 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:13.661670 (+ 2833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:43179: BlockingRecv error: recv got EOF from 127.27.40.130:43179 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 03:33:13 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:13 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:13.794008 9074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:13.789462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:56009)
0809 03:33:13.789635 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:13.789652 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:13.789710 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:13.789819 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:13.789819 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:13.790009 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:13.790060 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:13.790061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:13.790709 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:13.790710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:13.791266 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:13.791267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:13.791639 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:13.791640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:13.791682 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:13.792140 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:13.792145 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:13.793938 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":117,"threads_started":1}
W20260809 03:33:13.794096 9071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:13.789519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:56009 (local address 127.27.40.190:34409)
0809 03:33:13.789617 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:13.789619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:13.789709 (+ 90us) server_negotiation.cc:407] Connection header received
0809 03:33:13.789762 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:13.789762 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:13.789774 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:13.789814 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:13.790119 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:13.790633 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:13.791364 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:13.791552 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:13.791622 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:13.794046 (+ 2424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:56009: BlockingRecv error: recv got EOF from 127.27.40.131:56009 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 03:33:14 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:14 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:14.549450 9075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:14.544564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:34397)
0809 03:33:14.544771 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:14.544784 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:14.544846 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:14.544969 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:14.544969 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:14.545154 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:14.545207 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:14.545208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:14.546166 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:14.546167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:14.546714 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:14.546716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:14.547104 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:14.547105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:14.547151 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:14.547628 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:14.547633 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:14.549389 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":154,"threads_started":1}
W20260809 03:33:14.549588 9077 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:14.544666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:34397 (local address 127.27.40.190:34409)
0809 03:33:14.544815 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:14.544817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:14.544843 (+ 26us) server_negotiation.cc:407] Connection header received
0809 03:33:14.544899 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:14.544899 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:14.544911 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:14.544964 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:14.545270 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:14.546088 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:14.546817 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:14.547016 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:14.547112 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:14.549483 (+ 2371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:34397: BlockingRecv error: recv got EOF from 127.27.40.129:34397 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
Aug 09 03:33:14 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:14 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:14.667143 9078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:14.662174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:50169)
0809 03:33:14.662360 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:14.662371 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:14.662462 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:14.662598 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:14.662599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:14.662798 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:14.662845 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:14.662847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:14.663628 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:14.663630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:14.664184 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:14.664185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:14.664584 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:14.664585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:14.664644 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:14.665152 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:14.665159 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:14.667045 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":51,"threads_started":1}
W20260809 03:33:14.667202 9077 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:14.662272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:50169 (local address 127.27.40.190:34409)
0809 03:33:14.662334 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:14.662337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:14.662441 (+ 104us) server_negotiation.cc:407] Connection header received
0809 03:33:14.662531 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:14.662532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:14.662544 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:14.662595 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:14.662939 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:14.663542 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:14.664274 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:14.664494 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:14.664581 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:14.667131 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:50169: BlockingRecv error: recv got EOF from 127.27.40.130:50169 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 09 03:33:14 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:14 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:14.799737 9079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:14.794644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:34699)
0809 03:33:14.794815 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:14.794826 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:14.794912 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:14.795055 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:14.795055 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:14.795245 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:14.795297 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:14.795299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:14.796159 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:14.796161 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:14.796948 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:14.796950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:14.797326 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:14.797327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:14.797381 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:14.797862 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:14.797866 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:14.799677 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
W20260809 03:33:14.799839 9077 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:14.794717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:34699 (local address 127.27.40.190:34409)
0809 03:33:14.794815 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:14.794817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:14.794887 (+ 70us) server_negotiation.cc:407] Connection header received
0809 03:33:14.794984 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:14.794985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:14.794998 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:14.795048 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:14.795398 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:14.796018 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:14.797043 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:14.797249 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:14.797309 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:14.799770 (+ 2461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:34699: BlockingRecv error: recv got EOF from 127.27.40.131:34699 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 03:33:15 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:15 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:15.559255 9080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:15.550008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:43649)
0809 03:33:15.550177 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:15.550189 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:15.550250 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:15.550390 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:15.550391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:15.550564 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:15.550617 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:15.550619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:15.551270 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:15.551271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:15.551823 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:15.551824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:15.552150 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:15.552151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:15.552193 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:15.557185 (+ 4992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:15.557190 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:15.559140 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":97,"threads_started":1}
W20260809 03:33:15.559301 9082 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:15.550107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:43649 (local address 127.27.40.190:34409)
0809 03:33:15.550223 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:15.550225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:15.550235 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:33:15.550309 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:15.550309 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:15.550320 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:15.550370 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:15.550693 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:15.551200 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:15.551894 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:15.552082 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:15.552135 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:15.559225 (+ 7090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:43649: BlockingRecv error: recv got EOF from 127.27.40.129:43649 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":54,"threads_started":1}
Aug 09 03:33:15 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:15 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:15.672703 9083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:15.667703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:56779)
0809 03:33:15.667837 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:15.667848 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:15.667897 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:15.668048 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:15.668048 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:15.668229 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:15.668282 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:15.668284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:15.669027 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:15.669028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:15.669662 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:15.669664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:15.670024 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:15.670025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:15.670063 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:15.670541 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:15.670546 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:15.672643 (+ 2097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
W20260809 03:33:15.672768 9082 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:15.667826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:56779 (local address 127.27.40.190:34409)
0809 03:33:15.667895 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:15.667897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:15.667940 (+ 43us) server_negotiation.cc:407] Connection header received
0809 03:33:15.667966 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:15.667967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:15.667978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:15.668027 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:15.668397 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:15.668948 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:15.669746 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:15.669958 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:15.670015 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:15.672720 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:56779: BlockingRecv error: recv got EOF from 127.27.40.130:56779 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 03:33:15 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:15 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:15.804922 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:15.800307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:51783)
0809 03:33:15.800454 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:15.800469 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:15.800526 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:15.800663 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:15.800664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:15.800834 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:15.800885 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:15.800887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:15.801556 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:15.801557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:15.802111 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:15.802113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:15.802589 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:15.802590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:15.802684 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:15.803129 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:15.803135 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:15.804851 (+ 1716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
W20260809 03:33:15.804987 9082 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:15.800365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:51783 (local address 127.27.40.190:34409)
0809 03:33:15.800440 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:15.800441 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:15.800525 (+ 84us) server_negotiation.cc:407] Connection header received
0809 03:33:15.800570 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:15.800571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:15.800583 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:15.800641 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:15.800947 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:15.801469 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:15.802193 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:15.802495 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:15.802571 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:15.804939 (+ 2368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:51783: BlockingRecv error: recv got EOF from 127.27.40.131:51783 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 03:33:16 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:16 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:16.565249 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:16.559856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:51485)
0809 03:33:16.560021 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:16.560032 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:16.560095 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:16.560303 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:16.560304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:16.560482 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:16.560539 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:16.560541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:16.561297 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:16.561298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:16.561941 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:16.561943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:16.562458 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:16.562460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:16.562532 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:16.563101 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:16.563107 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:16.565151 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":99,"threads_started":1}
W20260809 03:33:16.565310 9087 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:16.559959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:51485 (local address 127.27.40.190:34409)
0809 03:33:16.560118 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:16.560120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:16.560127 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:33:16.560167 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:16.560168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:16.560178 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:16.560238 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:16.560628 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:16.561193 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:16.562021 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:16.562375 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:16.562441 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:16.565235 (+ 2794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:51485: BlockingRecv error: recv got EOF from 127.27.40.129:51485 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
Aug 09 03:33:16 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:16 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:16.678268 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:16.673278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:43227)
0809 03:33:16.673451 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:16.673481 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:16.673575 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:16.673762 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:16.673763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:16.673956 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:16.674013 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:16.674014 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:16.674724 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:16.674725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:16.675337 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:16.675339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:16.675723 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:16.675724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:16.675778 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:16.676264 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:16.676269 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:16.678196 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":103,"threads_started":1}
W20260809 03:33:16.678339 9087 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:16.673342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:43227 (local address 127.27.40.190:34409)
0809 03:33:16.673431 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:16.673433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:16.673553 (+ 120us) server_negotiation.cc:407] Connection header received
0809 03:33:16.673669 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:16.673670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:16.673681 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:16.673736 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:16.674105 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:16.674619 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:16.675421 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:16.675617 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:16.675683 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:16.678290 (+ 2607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:43227: BlockingRecv error: recv got EOF from 127.27.40.130:43227 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 03:33:16 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:16 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.131@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:16.810627 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:16.805430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.131:34325)
0809 03:33:16.805656 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:16.805670 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:16.805787 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:16.805973 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:16.805975 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:16.806197 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:16.806252 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:16.806254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:16.807025 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:16.807027 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:16.807645 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:16.807647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:16.808059 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:16.808061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:16.808134 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:16.808622 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:16.808627 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:16.810565 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":126,"threads_started":1}
W20260809 03:33:16.810856 9087 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:16.805569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:34325 (local address 127.27.40.190:34409)
0809 03:33:16.805650 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:16.805653 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:16.805771 (+ 118us) server_negotiation.cc:407] Connection header received
0809 03:33:16.805851 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:16.805853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:16.805878 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:16.805961 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:16.806351 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:16.806917 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:16.807744 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:16.807959 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:16.808057 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:16.810780 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.131:34325: BlockingRecv error: recv got EOF from 127.27.40.131:34325 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 09 03:33:17 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:17 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.129@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:17.570853 9092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:17.565830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.129:50691)
0809 03:33:17.566021 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:17.566032 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:17.566079 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:17.566308 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:17.566309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:17.566477 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:17.566534 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:17.566535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:17.567206 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:17.567207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:17.567730 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:17.567731 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:17.568179 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:17.568181 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:17.568229 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:17.568781 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:17.568786 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:17.570756 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":129,"threads_started":1}
W20260809 03:33:17.570919 9093 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:17.565928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:50691 (local address 127.27.40.190:34409)
0809 03:33:17.566071 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:17.566073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:17.566081 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:33:17.566192 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:17.566193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:17.566203 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:17.566268 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:17.566608 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:17.567136 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:17.567815 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:17.568091 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:17.568145 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:17.570843 (+ 2698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.129:50691: BlockingRecv error: recv got EOF from 127.27.40.129:50691 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
Aug 09 03:33:17 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:33:17 dist-test-slave-gkw7 krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.40.130@KRBTEST.COM for oryx/127.27.40.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:33:17.683555 9094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:17.678859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34409 (local address 127.27.40.130:34595)
0809 03:33:17.679006 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:17.679016 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:17.679085 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:17.679193 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:17.679194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:17.679369 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:17.679420 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:17.679421 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:17.680034 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:17.680034 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:17.680643 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:17.680646 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:17.681017 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:17.681018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:17.681090 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:17.681566 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:17.681572 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:17.683497 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.40.190:34409: Server oryx/127.27.40.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":59,"threads_started":1}
W20260809 03:33:17.683686 9093 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:17.678945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:34595 (local address 127.27.40.190:34409)
0809 03:33:17.678997 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:17.678999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:17.679098 (+ 99us) server_negotiation.cc:407] Connection header received
0809 03:33:17.679128 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:17.679129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:17.679140 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:17.679182 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:17.679484 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:17.679967 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:17.680728 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:17.680929 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:17.681015 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:17.683606 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.40.130:34595: BlockingRecv error: recv got EOF from 127.27.40.130:34595 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 03:33:17.723536 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 8413
I20260809 03:33:17.737134 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 8641
I20260809 03:33:17.750427 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 8823
I20260809 03:33:17.757889 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 8319
2026-08-09T03:33:17Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17725 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T03:33:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:33:17Z Disabled control of system clock
I20260809 03:33:17.788668 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:34505
--webserver_interface=127.27.40.190
--webserver_port=0
--builtin_ntp_servers=127.27.40.148:39547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:34505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 03:33:17.870471 9100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:17.870714 9100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:17.870755 9100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:17.871943 9100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:33:17.872009 9100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:17.872031 9100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:17.872063 9100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:33:17.872112 9100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:33:17.873449 9100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:34505
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:34505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:17.873742 9100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:17.874027 9100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:17.878857 9107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:17.878896 9108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:17.878890 9112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:17.879124 9100 server_base.cc:1034] running on GCE node
I20260809 03:33:17.879360 9100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:17.879674 9100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:17.880846 9100 hybrid_clock.cc:648] HybridClock initialized: now 1786246397880828 us; error 31 us; skew 500 ppm
I20260809 03:33:17.882141 9100 webserver.cc:460] Webserver started at http://127.27.40.190:33609/ using document root <none> and password file <none>
I20260809 03:33:17.882380 9100 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:17.882469 9100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:17.882637 9100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:33:17.883682 9100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "d99ea67ece2e4e49b320eab48317b8bf"
format_stamp: "Formatted at 2026-08-09 03:33:17 on dist-test-slave-gkw7"
server_key: "c433763bc1282678b5ef469f756cd696"
server_key_iv: "84435fe249f5b4d55e26382fe645ac05"
server_key_version: "encryptionkey@0"
I20260809 03:33:17.884048 9100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "d99ea67ece2e4e49b320eab48317b8bf"
format_stamp: "Formatted at 2026-08-09 03:33:17 on dist-test-slave-gkw7"
server_key: "c433763bc1282678b5ef469f756cd696"
server_key_iv: "84435fe249f5b4d55e26382fe645ac05"
server_key_version: "encryptionkey@0"
I20260809 03:33:17.885444 9100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:33:17.886327 9120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:17.886514 9100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:33:17.886631 9100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "d99ea67ece2e4e49b320eab48317b8bf"
format_stamp: "Formatted at 2026-08-09 03:33:17 on dist-test-slave-gkw7"
server_key: "c433763bc1282678b5ef469f756cd696"
server_key_iv: "84435fe249f5b4d55e26382fe645ac05"
server_key_version: "encryptionkey@0"
I20260809 03:33:17.886723 9100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:17.905153 9100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:17.905514 9100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:17.905751 9100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:17.910194 9100 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:34505
I20260809 03:33:17.910230 9195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:34505 every 8 connection(s)
I20260809 03:33:17.910619 9100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:33:17.911324 9196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:33:17.912964 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 9100
I20260809 03:33:17.913130 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:33:17.913337 27810 external_mini_cluster.cc:1428] Setting key ee195c11eb020c529fc56cb55f46fcbc
I20260809 03:33:17.914410 9196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf: Bootstrap starting.
I20260809 03:33:17.915028 9196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf: Neither blocks nor log segments found. Creating new log.
I20260809 03:33:17.915330 9196 log.cc:824] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf: Log is configured to *not* fsync() on all Append() calls
I20260809 03:33:17.916033 9196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf: No bootstrap required, opened a new log
I20260809 03:33:17.917834 9196 raft_consensus.cc:348] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99ea67ece2e4e49b320eab48317b8bf" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34505 } }
I20260809 03:33:17.917990 9196 raft_consensus.cc:374] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:33:17.918054 9196 raft_consensus.cc:729] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d99ea67ece2e4e49b320eab48317b8bf, State: Initialized, Role: FOLLOWER
I20260809 03:33:17.918247 9196 consensus_queue.cc:260] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99ea67ece2e4e49b320eab48317b8bf" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34505 } }
I20260809 03:33:17.918346 9196 raft_consensus.cc:388] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:33:17.918411 9196 raft_consensus.cc:482] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:33:17.918464 9196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:33:17.919018 9196 raft_consensus.cc:504] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99ea67ece2e4e49b320eab48317b8bf" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34505 } }
I20260809 03:33:17.919148 9196 leader_election.cc:302] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d99ea67ece2e4e49b320eab48317b8bf; no voters:
I20260809 03:33:17.919350 9196 leader_election.cc:288] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:33:17.919385 9201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:33:17.919451 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:17.913974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33786 (local address 127.27.40.190:34505)
0809 03:33:17.914212 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:17.914220 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:17.914247 (+ 27us) server_negotiation.cc:407] Connection header received
0809 03:33:17.914768 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:17.914776 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:17.914843 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:17.915005 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:17.915253 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:17.916376 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:17.917226 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:17.917743 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:17.917922 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:17.918706 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:17.918710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:17.918711 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:33:17.918898 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:17.918933 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:33:17.918944 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:17.919023 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:17.919125 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 03:33:17.919213 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":82,"threads_started":1}
I20260809 03:33:17.919516 9201 raft_consensus.cc:686] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [term 1 LEADER]: Becoming Leader. State: Replica: d99ea67ece2e4e49b320eab48317b8bf, State: Running, Role: LEADER
I20260809 03:33:17.919798 9201 consensus_queue.cc:237] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99ea67ece2e4e49b320eab48317b8bf" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34505 } }
I20260809 03:33:17.919639 9196 sys_catalog.cc:564] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:33:17.920725 9203 sys_catalog.cc:455] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d99ea67ece2e4e49b320eab48317b8bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99ea67ece2e4e49b320eab48317b8bf" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34505 } } }
I20260809 03:33:17.920744 9202 sys_catalog.cc:455] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader d99ea67ece2e4e49b320eab48317b8bf. Latest consensus state: current_term: 1 leader_uuid: "d99ea67ece2e4e49b320eab48317b8bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99ea67ece2e4e49b320eab48317b8bf" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34505 } } }
I20260809 03:33:17.920866 9202 sys_catalog.cc:458] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [sys.catalog]: This master's current role is: LEADER
I20260809 03:33:17.920818 9203 sys_catalog.cc:458] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf [sys.catalog]: This master's current role is: LEADER
I20260809 03:33:17.921070 9214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:33:17.921622 9214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:33:17.923249 9214 catalog_manager.cc:1347] Generated new cluster ID: f778a2d51a654dd9b36a25120ad9739f
I20260809 03:33:17.923307 9214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:33:17.930513 9214 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:33:17.931061 9214 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:33:17.941030 9214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d99ea67ece2e4e49b320eab48317b8bf: Generated new TSK 0
I20260809 03:33:17.941251 9214 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:33:17.944816 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34505
--builtin_ntp_servers=127.27.40.148:39547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 03:33:18.046482 9226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:18.046716 9226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:18.046741 9226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:18.047955 9226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:18.048000 9226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:18.048025 9226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:33:18.049394 9226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:18.049678 9226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:18.050027 9226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:18.056516 9235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:18.056505 9238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:18.056723 9231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:18.056766 9226 server_base.cc:1034] running on GCE node
I20260809 03:33:18.057018 9226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:18.057307 9226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:18.058462 9226 hybrid_clock.cc:648] HybridClock initialized: now 1786246398058441 us; error 40 us; skew 500 ppm
I20260809 03:33:18.059873 9226 webserver.cc:460] Webserver started at http://127.27.40.129:41421/ using document root <none> and password file <none>
I20260809 03:33:18.060135 9226 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:18.060187 9226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:18.060324 9226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:33:18.061421 9226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "d0e73c3c7a6541ae94fcda822183463b"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "a9c463d901eecc2e1b45e8bf3376733d"
server_key_iv: "6b15c63d6273373dde29f6b39d0b535c"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.061880 9226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "d0e73c3c7a6541ae94fcda822183463b"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "a9c463d901eecc2e1b45e8bf3376733d"
server_key_iv: "6b15c63d6273373dde29f6b39d0b535c"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.063261 9226 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:33:18.064055 9244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.064222 9226 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:33:18.064329 9226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "d0e73c3c7a6541ae94fcda822183463b"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "a9c463d901eecc2e1b45e8bf3376733d"
server_key_iv: "6b15c63d6273373dde29f6b39d0b535c"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.064405 9226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:18.083802 9226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:18.084133 9226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:18.084295 9226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:18.084551 9226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:33:18.084923 9226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:33:18.084980 9226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.085017 9226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:33:18.085047 9226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.091411 9226 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:45539
I20260809 03:33:18.091456 9414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:45539 every 8 connection(s)
I20260809 03:33:18.091850 9226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:33:18.097059 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:18.092630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:60693 (local address 127.27.40.190:34505)
0809 03:33:18.092729 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:18.092731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:18.092922 (+ 191us) server_negotiation.cc:407] Connection header received
0809 03:33:18.093410 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:18.093411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:18.093420 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:18.093469 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:18.094209 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:18.094705 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:18.095701 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:18.095884 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:18.095938 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:18.096771 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:18.096773 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:18.096773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:33:18.096778 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:18.096791 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:33:18.096795 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:18.096837 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:18.096968 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 03:33:18.097000 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 03:33:18.097307 9420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:18.092583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34505 (local address 127.27.40.129:60693)
0809 03:33:18.092802 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:18.092836 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:18.093327 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:18.093629 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:18.093633 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:18.093642 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:18.094131 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:18.094136 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:18.094791 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:18.094791 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:18.095610 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:18.095612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:18.095962 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:18.095963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:18.096073 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:18.096629 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:33:18.096629 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:18.096658 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:33:18.096681 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:33:18.096854 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:33:18.096859 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:33:18.096862 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:33:18.096919 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 03:33:18.097065 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":113,"threads_started":1}
I20260809 03:33:18.097827 9417 heartbeater.cc:347] Connected to a master server at 127.27.40.190:34505
I20260809 03:33:18.097916 9417 heartbeater.cc:464] Registering TS with master...
I20260809 03:33:18.098165 9417 heartbeater.cc:511] Master 127.27.40.190:34505 requested a full tablet report, sending...
I20260809 03:33:18.098690 9144 ts_manager.cc:194] Registered new tserver with Master: d0e73c3c7a6541ae94fcda822183463b (127.27.40.129:45539)
I20260809 03:33:18.099311 9144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.40.129:60693
I20260809 03:33:18.101884 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 9226
I20260809 03:33:18.101961 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:33:18.102203 27810 external_mini_cluster.cc:1428] Setting key 83ee49f32bc4e604316fc295195c5917
I20260809 03:33:18.103132 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34505
--builtin_ntp_servers=127.27.40.148:39547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 03:33:18.184939 9422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:18.185146 9422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:18.185168 9422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:18.186221 9422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:18.186254 9422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:18.186278 9422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:33:18.187479 9422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:18.187644 9422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:18.187866 9422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:18.193041 9434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:18.193001 9429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:18.193284 9422 server_base.cc:1034] running on GCE node
W20260809 03:33:18.192998 9428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:18.193611 9422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:18.193889 9422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:18.195040 9422 hybrid_clock.cc:648] HybridClock initialized: now 1786246398195029 us; error 28 us; skew 500 ppm
I20260809 03:33:18.196231 9422 webserver.cc:460] Webserver started at http://127.27.40.130:44307/ using document root <none> and password file <none>
I20260809 03:33:18.196485 9422 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:18.196550 9422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:18.196691 9422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:33:18.197763 9422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "4497cc5bdc3c48c8b1418c8055f91578"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "7e9cdcb5c81a7fd20771d0821b1aeb61"
server_key_iv: "be697ba130c6aa2accac920029436461"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.198102 9422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "4497cc5bdc3c48c8b1418c8055f91578"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "7e9cdcb5c81a7fd20771d0821b1aeb61"
server_key_iv: "be697ba130c6aa2accac920029436461"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.199462 9422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:33:18.200181 9441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.200366 9422 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:33:18.200459 9422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "4497cc5bdc3c48c8b1418c8055f91578"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "7e9cdcb5c81a7fd20771d0821b1aeb61"
server_key_iv: "be697ba130c6aa2accac920029436461"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.200536 9422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:18.229655 9422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:18.229977 9422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:18.230135 9422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:18.230381 9422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:33:18.230715 9422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:33:18.230768 9422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.230803 9422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:33:18.230829 9422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.237285 9422 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:39217
I20260809 03:33:18.237353 9608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:39217 every 8 connection(s)
I20260809 03:33:18.237758 9422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:33:18.238864 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 9422
I20260809 03:33:18.238999 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:33:18.239202 27810 external_mini_cluster.cc:1428] Setting key 54b6f69fe23055f82d5bfaa83130c14b
I20260809 03:33:18.240360 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34505
--builtin_ntp_servers=127.27.40.148:39547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 03:33:18.243803 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:18.238600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:45235 (local address 127.27.40.190:34505)
0809 03:33:18.238692 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:18.238698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:18.238956 (+ 258us) server_negotiation.cc:407] Connection header received
0809 03:33:18.239619 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:18.239621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:18.239636 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:18.239713 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:18.240452 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:18.241208 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:18.242348 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:18.242554 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:18.242644 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:18.243519 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:18.243521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:18.243521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:33:18.243526 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:18.243542 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:33:18.243547 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:18.243605 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:18.243693 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 03:33:18.243723 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 03:33:18.244025 9614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:18.238596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34505 (local address 127.27.40.130:45235)
0809 03:33:18.238831 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:18.238865 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:18.239495 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:18.239767 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:18.239770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:18.239779 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:18.240354 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:18.240359 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:18.241327 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:18.241328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:18.242221 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:18.242224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:18.242654 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:18.242655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:18.242790 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:18.243372 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:33:18.243373 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:18.243402 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:33:18.243424 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:33:18.243596 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:33:18.243602 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:33:18.243605 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:33:18.243695 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 03:33:18.243800 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":115,"threads_started":1}
I20260809 03:33:18.244580 9611 heartbeater.cc:347] Connected to a master server at 127.27.40.190:34505
I20260809 03:33:18.244690 9611 heartbeater.cc:464] Registering TS with master...
I20260809 03:33:18.244882 9611 heartbeater.cc:511] Master 127.27.40.190:34505 requested a full tablet report, sending...
I20260809 03:33:18.245321 9144 ts_manager.cc:194] Registered new tserver with Master: 4497cc5bdc3c48c8b1418c8055f91578 (127.27.40.130:39217)
I20260809 03:33:18.245908 9144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.40.130:45235
W20260809 03:33:18.326566 9615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:18.326776 9615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:18.326798 9615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:18.327889 9615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:18.327929 9615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:18.327956 9615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:33:18.329368 9615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:34505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:18.329625 9615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:18.329895 9615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:18.334897 9621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:18.334908 9624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:18.334941 9622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:18.335258 9615 server_base.cc:1034] running on GCE node
I20260809 03:33:18.335476 9615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:18.335716 9615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:18.336882 9615 hybrid_clock.cc:648] HybridClock initialized: now 1786246398336855 us; error 37 us; skew 500 ppm
I20260809 03:33:18.338239 9615 webserver.cc:460] Webserver started at http://127.27.40.131:33727/ using document root <none> and password file <none>
I20260809 03:33:18.338503 9615 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:18.338574 9615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:18.338738 9615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:33:18.339780 9615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "9360dad92f534ceb8a41945d2212b455"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "cbca944519d3ae2d7c28e3517dd7cc1d"
server_key_iv: "225d3c3a7dd89fd3df1c8ed4b4ea088d"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.340143 9615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "9360dad92f534ceb8a41945d2212b455"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "cbca944519d3ae2d7c28e3517dd7cc1d"
server_key_iv: "225d3c3a7dd89fd3df1c8ed4b4ea088d"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.341470 9615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:33:18.342218 9632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.342397 9615 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:33:18.342496 9615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "9360dad92f534ceb8a41945d2212b455"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "cbca944519d3ae2d7c28e3517dd7cc1d"
server_key_iv: "225d3c3a7dd89fd3df1c8ed4b4ea088d"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.342577 9615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:18.358546 9615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:18.358863 9615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:18.359025 9615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:18.359261 9615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:33:18.359601 9615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:33:18.359660 9615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.359704 9615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:33:18.359750 9615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.366343 9615 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:37579
I20260809 03:33:18.366366 9792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:37579 every 8 connection(s)
I20260809 03:33:18.366744 9615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:33:18.372301 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:18.367578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:51111 (local address 127.27.40.190:34505)
0809 03:33:18.367650 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:18.367652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:18.367895 (+ 243us) server_negotiation.cc:407] Connection header received
0809 03:33:18.368477 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:18.368479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:18.368491 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:18.368542 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:18.369211 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:18.369853 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:18.370844 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:18.371061 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:18.371140 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:18.372017 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:18.372019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:18.372019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:33:18.372024 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:18.372038 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:33:18.372042 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:18.372093 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:18.372194 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 03:33:18.372222 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 03:33:18.372510 9795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:18.367561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:34505 (local address 127.27.40.131:51111)
0809 03:33:18.367774 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:18.367806 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:18.368369 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:18.368609 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:18.368613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:18.368622 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:18.369116 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:18.369121 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:18.369952 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:18.369953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:18.370752 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:18.370755 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:18.371160 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:18.371161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:18.371295 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:18.371864 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:33:18.371864 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:18.371892 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:33:18.371914 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:33:18.372104 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:33:18.372109 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:33:18.372112 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:33:18.372195 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 03:33:18.372295 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260809 03:33:18.373047 9793 heartbeater.cc:347] Connected to a master server at 127.27.40.190:34505
I20260809 03:33:18.373126 9793 heartbeater.cc:464] Registering TS with master...
I20260809 03:33:18.373262 9793 heartbeater.cc:511] Master 127.27.40.190:34505 requested a full tablet report, sending...
I20260809 03:33:18.373629 9144 ts_manager.cc:194] Registered new tserver with Master: 9360dad92f534ceb8a41945d2212b455 (127.27.40.131:37579)
I20260809 03:33:18.374009 9144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.40.131:51111
I20260809 03:33:18.376535 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 9615
I20260809 03:33:18.376628 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:33:18.376842 27810 external_mini_cluster.cc:1428] Setting key e1e0be6f33f984075602c97b57fde637
I20260809 03:33:18.377822 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 03:33:18.379679 9199 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:18.379154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33792 (local address 127.27.40.190:34505)
0809 03:33:18.379212 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:18.379214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:18.379360 (+ 146us) server_negotiation.cc:407] Connection header received
0809 03:33:18.379433 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:18.379434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:18.379444 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:18.379498 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:18.379617 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33792: BlockingRecv error: recv got EOF from 127.0.0.1:33792 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 03:33:18.379751 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 9226
I20260809 03:33:18.402108 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 9422
I20260809 03:33:18.409953 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 9615
I20260809 03:33:18.418429 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 9100
2026-08-09T03:33:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (661 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T03:33:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:33:18Z Disabled control of system clock
I20260809 03:33:18.449514 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:33079
--webserver_interface=127.27.40.190
--webserver_port=0
--builtin_ntp_servers=127.27.40.148:41615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:33079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 03:33:18.533377 9810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:18.533596 9810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:18.533617 9810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:18.534649 9810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:33:18.534687 9810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:18.534700 9810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:18.534713 9810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:33:18.534724 9810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:33:18.535930 9810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:41615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:33079
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:33079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:18.536104 9810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:18.536322 9810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:18.541067 9818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:18.541168 9810 server_base.cc:1034] running on GCE node
W20260809 03:33:18.541109 9816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:18.541087 9815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:18.541605 9810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:18.541945 9810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:18.543083 9810 hybrid_clock.cc:648] HybridClock initialized: now 1786246398543088 us; error 46 us; skew 500 ppm
I20260809 03:33:18.544425 9810 webserver.cc:460] Webserver started at http://127.27.40.190:44467/ using document root <none> and password file <none>
I20260809 03:33:18.544703 9810 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:18.544749 9810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:18.544916 9810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:33:18.546000 9810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "712f109fee8b40e191083a52fb851768"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "c9cb0c0603424f5ff8d2387125d0db52"
server_key_iv: "9bd906339c895139758f2a9b24eef899"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.546358 9810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "712f109fee8b40e191083a52fb851768"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "c9cb0c0603424f5ff8d2387125d0db52"
server_key_iv: "9bd906339c895139758f2a9b24eef899"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.547681 9810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:33:18.548468 9826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.548693 9810 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:33:18.548765 9810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "712f109fee8b40e191083a52fb851768"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "c9cb0c0603424f5ff8d2387125d0db52"
server_key_iv: "9bd906339c895139758f2a9b24eef899"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.548853 9810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:18.552139 9810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:18.552397 9810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:18.552532 9810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:18.556725 9810 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:33079
I20260809 03:33:18.556757 9897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:33079 every 8 connection(s)
I20260809 03:33:18.557065 9810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:33:18.557762 9898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:33:18.560690 9898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768: Bootstrap starting.
I20260809 03:33:18.561234 9898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768: Neither blocks nor log segments found. Creating new log.
I20260809 03:33:18.561470 9898 log.cc:824] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768: Log is configured to *not* fsync() on all Append() calls
I20260809 03:33:18.562211 9898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768: No bootstrap required, opened a new log
I20260809 03:33:18.563716 9898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712f109fee8b40e191083a52fb851768" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 33079 } }
I20260809 03:33:18.563846 9898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:33:18.563864 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 9810
I20260809 03:33:18.563868 9898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 712f109fee8b40e191083a52fb851768, State: Initialized, Role: FOLLOWER
I20260809 03:33:18.563970 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:33:18.564024 9898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712f109fee8b40e191083a52fb851768" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 33079 } }
I20260809 03:33:18.564131 9898 raft_consensus.cc:388] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:33:18.564172 9898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:33:18.564193 27810 external_mini_cluster.cc:1428] Setting key e3e1262c29686575d2f8125b0ffaf178
I20260809 03:33:18.564219 9898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:33:18.565004 9898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712f109fee8b40e191083a52fb851768" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 33079 } }
I20260809 03:33:18.565121 9898 leader_election.cc:302] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 712f109fee8b40e191083a52fb851768; no voters:
I20260809 03:33:18.565275 9898 leader_election.cc:288] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:33:18.565333 9905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:33:18.565662 9898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:33:18.565898 9905 raft_consensus.cc:686] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [term 1 LEADER]: Becoming Leader. State: Replica: 712f109fee8b40e191083a52fb851768, State: Running, Role: LEADER
I20260809 03:33:18.566039 9905 consensus_queue.cc:237] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712f109fee8b40e191083a52fb851768" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 33079 } }
I20260809 03:33:18.566649 9907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "712f109fee8b40e191083a52fb851768" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712f109fee8b40e191083a52fb851768" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 33079 } } }
I20260809 03:33:18.566690 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:18.564702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60922 (local address 127.27.40.190:33079)
0809 03:33:18.564933 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:18.564942 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:18.564978 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:33:18.565089 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:18.565096 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:18.565141 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:18.565293 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:18.565992 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:18.565995 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:18.565996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:33:18.566001 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:18.566050 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:33:18.566067 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:18.566146 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:18.566250 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 03:33:18.566440 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":92,"threads_started":1}
I20260809 03:33:18.566730 9907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [sys.catalog]: This master's current role is: LEADER
I20260809 03:33:18.566917 9908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 712f109fee8b40e191083a52fb851768. Latest consensus state: current_term: 1 leader_uuid: "712f109fee8b40e191083a52fb851768" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712f109fee8b40e191083a52fb851768" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 33079 } } }
I20260809 03:33:18.566988 9908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768 [sys.catalog]: This master's current role is: LEADER
I20260809 03:33:18.567281 9918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:33:18.567806 9918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:33:18.569249 9918 catalog_manager.cc:1347] Generated new cluster ID: 6e7ce3536ced496aa62a6b533fe05e5d
I20260809 03:33:18.569335 9918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:33:18.582547 9918 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:33:18.582728 9918 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:33:18.585423 9918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 712f109fee8b40e191083a52fb851768: Generated new TSK 0
I20260809 03:33:18.585628 9918 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:33:18.595975 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:33079
--builtin_ntp_servers=127.27.40.148:41615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 03:33:18.680300 9934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:18.680512 9934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:18.680536 9934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:18.681865 9934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:18.681910 9934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:18.681936 9934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:33:18.683302 9934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:41615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:33079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:18.683574 9934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:18.683898 9934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:18.689059 9944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:18.689054 9942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:18.689051 9941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:18.689302 9934 server_base.cc:1034] running on GCE node
I20260809 03:33:18.689581 9934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:18.689850 9934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:18.691025 9934 hybrid_clock.cc:648] HybridClock initialized: now 1786246398690993 us; error 44 us; skew 500 ppm
I20260809 03:33:18.692399 9934 webserver.cc:460] Webserver started at http://127.27.40.129:33419/ using document root <none> and password file <none>
I20260809 03:33:18.692659 9934 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:18.692734 9934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:18.692888 9934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:33:18.694070 9934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "841abe5ecbab4600910c72af84283923"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "997ac246fcde9649c091edf24acdfac0"
server_key_iv: "64cb1904750267177650e2d6bc4ba395"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.694450 9934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "841abe5ecbab4600910c72af84283923"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "997ac246fcde9649c091edf24acdfac0"
server_key_iv: "64cb1904750267177650e2d6bc4ba395"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.695838 9934 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:33:18.696655 9955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.696857 9934 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:33:18.696966 9934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "841abe5ecbab4600910c72af84283923"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "997ac246fcde9649c091edf24acdfac0"
server_key_iv: "64cb1904750267177650e2d6bc4ba395"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.697041 9934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:18.700555 9934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:18.700843 9934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:18.700991 9934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:18.701227 9934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:33:18.701592 9934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:33:18.701653 9934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.701694 9934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:33:18.701735 9934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.707734 9934 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:40883
I20260809 03:33:18.707787 10118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:40883 every 8 connection(s)
I20260809 03:33:18.708135 9934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:33:18.711207 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:18.708891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:59945 (local address 127.27.40.190:33079)
0809 03:33:18.708936 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:18.708939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:18.709166 (+ 227us) server_negotiation.cc:407] Connection header received
0809 03:33:18.709697 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:18.709698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:18.709705 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:18.709759 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:18.710963 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:18.710965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:18.710965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:33:18.710971 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:18.710991 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:33:18.710996 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:18.711084 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:18.711131 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 03:33:18.711147 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 03:33:18.711417 10123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:18.708866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:33079 (local address 127.27.40.129:59945)
0809 03:33:18.709062 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:18.709093 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:18.709587 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:18.709813 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:18.709816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:18.709823 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:18.710799 (+ 976us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:33:18.710799 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:18.710826 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:33:18.710854 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:33:18.711081 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:33:18.711084 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:33:18.711086 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 03:33:18.711137 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 03:33:18.711183 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":113,"threads_started":1}
I20260809 03:33:18.711759 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 9934
I20260809 03:33:18.711879 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:33:18.712067 27810 external_mini_cluster.cc:1428] Setting key b350e86cd6f4bc63eabbc7d860e7d0ea
I20260809 03:33:18.712069 10119 heartbeater.cc:347] Connected to a master server at 127.27.40.190:33079
I20260809 03:33:18.712224 10119 heartbeater.cc:464] Registering TS with master...
I20260809 03:33:18.712379 10119 heartbeater.cc:511] Master 127.27.40.190:33079 requested a full tablet report, sending...
I20260809 03:33:18.712922 9851 ts_manager.cc:194] Registered new tserver with Master: 841abe5ecbab4600910c72af84283923 (127.27.40.129:40883)
I20260809 03:33:18.713073 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:33079
--builtin_ntp_servers=127.27.40.148:41615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 03:33:18.795890 10124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:18.796089 10124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:18.796113 10124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:18.797196 10124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:18.797232 10124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:18.797257 10124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:33:18.798671 10124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:41615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:33079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:18.798851 10124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:18.799099 10124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:18.804422 10131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:18.804440 10137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:18.804440 10133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:18.804535 10124 server_base.cc:1034] running on GCE node
I20260809 03:33:18.804850 10124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:18.805100 10124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:18.806272 10124 hybrid_clock.cc:648] HybridClock initialized: now 1786246398806253 us; error 36 us; skew 500 ppm
I20260809 03:33:18.807520 10124 webserver.cc:460] Webserver started at http://127.27.40.130:40037/ using document root <none> and password file <none>
I20260809 03:33:18.807793 10124 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:18.807878 10124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:18.808035 10124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:33:18.809136 10124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "315e25706c3143e1b0d9ce16988947fb"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "3868c59f0de1832878b13d6fb902702a"
server_key_iv: "72fb618cff61b9219f9a52ea6fc12828"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.809504 10124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "315e25706c3143e1b0d9ce16988947fb"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "3868c59f0de1832878b13d6fb902702a"
server_key_iv: "72fb618cff61b9219f9a52ea6fc12828"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.810866 10124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:33:18.811694 10144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.811872 10124 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:33:18.811965 10124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "315e25706c3143e1b0d9ce16988947fb"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "3868c59f0de1832878b13d6fb902702a"
server_key_iv: "72fb618cff61b9219f9a52ea6fc12828"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.812054 10124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:18.815513 10124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:18.815781 10124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:18.815924 10124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:18.816148 10124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:33:18.816463 10124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:33:18.816520 10124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.816560 10124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:33:18.816588 10124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.822469 10124 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:35519
I20260809 03:33:18.822516 10308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:35519 every 8 connection(s)
I20260809 03:33:18.822829 10124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:33:18.825856 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:18.823572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:37411 (local address 127.27.40.190:33079)
0809 03:33:18.823620 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:18.823622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:18.823889 (+ 267us) server_negotiation.cc:407] Connection header received
0809 03:33:18.824334 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:18.824335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:18.824341 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:18.824409 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:18.825645 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:18.825647 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:18.825647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:33:18.825652 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:18.825668 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:33:18.825672 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:18.825717 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:18.825786 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 03:33:18.825800 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:33:18.826045 10313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:18.823536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:33079 (local address 127.27.40.130:37411)
0809 03:33:18.823730 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:18.823773 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:18.824231 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:18.824462 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:18.824465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:18.824474 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:18.825419 (+ 945us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:33:18.825419 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:18.825449 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:33:18.825481 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:33:18.825716 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:33:18.825719 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:33:18.825722 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:33:18.825781 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 03:33:18.825833 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260809 03:33:18.826599 10309 heartbeater.cc:347] Connected to a master server at 127.27.40.190:33079
I20260809 03:33:18.826711 10309 heartbeater.cc:464] Registering TS with master...
I20260809 03:33:18.826869 10309 heartbeater.cc:511] Master 127.27.40.190:33079 requested a full tablet report, sending...
I20260809 03:33:18.827297 9851 ts_manager.cc:194] Registered new tserver with Master: 315e25706c3143e1b0d9ce16988947fb (127.27.40.130:35519)
I20260809 03:33:18.828639 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 10124
I20260809 03:33:18.828720 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:33:18.828925 27810 external_mini_cluster.cc:1428] Setting key 1242efb527cba902529b174593285a00
I20260809 03:33:18.829938 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:33079
--builtin_ntp_servers=127.27.40.148:41615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 03:33:18.913304 10314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:18.913506 10314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:18.913594 10314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:18.914785 10314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:18.914825 10314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:18.914850 10314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:33:18.916226 10314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:41615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:33079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:18.916419 10314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:18.916676 10314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:18.921640 10321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:18.921646 10322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:18.921653 10324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:18.921936 10314 server_base.cc:1034] running on GCE node
I20260809 03:33:18.922179 10314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:18.922418 10314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:18.923571 10314 hybrid_clock.cc:648] HybridClock initialized: now 1786246398923554 us; error 31 us; skew 500 ppm
I20260809 03:33:18.924866 10314 webserver.cc:460] Webserver started at http://127.27.40.131:35271/ using document root <none> and password file <none>
I20260809 03:33:18.925140 10314 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:18.925213 10314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:18.925379 10314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:33:18.926541 10314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "b5aafb969fbe4f819f7380c0ac1befe8"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "f1f04a1eeb02093b6e18c0d5b0f97538"
server_key_iv: "a07c0a2266cc776a3d93a0936b4debf2"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.926919 10314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "b5aafb969fbe4f819f7380c0ac1befe8"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "f1f04a1eeb02093b6e18c0d5b0f97538"
server_key_iv: "a07c0a2266cc776a3d93a0936b4debf2"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.928264 10314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:33:18.928985 10331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.929162 10314 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:33:18.929263 10314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "b5aafb969fbe4f819f7380c0ac1befe8"
format_stamp: "Formatted at 2026-08-09 03:33:18 on dist-test-slave-gkw7"
server_key: "f1f04a1eeb02093b6e18c0d5b0f97538"
server_key_iv: "a07c0a2266cc776a3d93a0936b4debf2"
server_key_version: "encryptionkey@0"
I20260809 03:33:18.929347 10314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:18.932652 10314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:18.932919 10314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:18.933073 10314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:18.933275 10314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:33:18.933635 10314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:33:18.933698 10314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.933737 10314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:33:18.933765 10314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:18.939915 10314 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:38597
I20260809 03:33:18.939960 10498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:38597 every 8 connection(s)
I20260809 03:33:18.940264 10314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:33:18.943536 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:18.941017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:40913 (local address 127.27.40.190:33079)
0809 03:33:18.941071 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:18.941074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:18.941324 (+ 250us) server_negotiation.cc:407] Connection header received
0809 03:33:18.941950 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:18.941951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:18.941959 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:18.942031 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:18.943250 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:18.943252 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:18.943252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:33:18.943257 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:18.943273 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:33:18.943278 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:18.943332 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:18.943441 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 03:33:18.943458 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 03:33:18.943761 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:18.940981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:33079 (local address 127.27.40.131:40913)
0809 03:33:18.941195 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:18.941229 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:18.941833 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:18.942117 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:18.942121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:18.942131 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:18.943086 (+ 955us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:33:18.943086 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:18.943116 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:33:18.943146 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:33:18.943341 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:33:18.943345 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:33:18.943349 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:33:18.943460 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 03:33:18.943526 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":84,"threads_started":1}
I20260809 03:33:18.944257 10499 heartbeater.cc:347] Connected to a master server at 127.27.40.190:33079
I20260809 03:33:18.944339 10499 heartbeater.cc:464] Registering TS with master...
I20260809 03:33:18.944455 10499 heartbeater.cc:511] Master 127.27.40.190:33079 requested a full tablet report, sending...
I20260809 03:33:18.944867 9851 ts_manager.cc:194] Registered new tserver with Master: b5aafb969fbe4f819f7380c0ac1befe8 (127.27.40.131:38597)
I20260809 03:33:18.945659 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 10314
I20260809 03:33:18.945740 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:33:18.945876 27810 external_mini_cluster.cc:1428] Setting key dbda6034c12823114432eaff9ad35f12
I20260809 03:33:18.946694 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 03:33:18.948534 9904 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:33:18.948017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60938 (local address 127.27.40.190:33079)
0809 03:33:18.948054 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:18.948056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:18.948196 (+ 140us) server_negotiation.cc:407] Connection header received
0809 03:33:18.948273 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:18.948274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:18.948279 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:18.948354 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:18.948481 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60938: BlockingRecv error: recv got EOF from 127.0.0.1:60938 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 03:33:18.948702 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 9934
I20260809 03:33:18.967293 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 10124
I20260809 03:33:18.980062 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 10314
I20260809 03:33:18.988555 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 9810
2026-08-09T03:33:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (576 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:33:19 dist-test-slave-gkw7 krb5kdc[10516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:33:19 dist-test-slave-gkw7 krb5kdc[10516](info): set up 2 sockets
Aug 09 03:33:19 dist-test-slave-gkw7 krb5kdc[10516](info): commencing operation
krb5kdc: starting...
W20260809 03:33:20.255132 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:33:20 dist-test-slave-gkw7 krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:33:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:33:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:33:20.385488 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:40663
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:43145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:40663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:33:20.476774 10538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:20.476971 10538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:20.476992 10538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:20.478082 10538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:33:20.478119 10538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:20.478133 10538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:20.478147 10538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:33:20.478159 10538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:33:20.479564 10538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:43145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:40663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:40663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:20.479758 10538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:20.479924 10538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:20.484812 10547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:20.484819 10543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:20.484819 10545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:20.485056 10538 server_base.cc:1034] running on GCE node
I20260809 03:33:20.485311 10538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:20.485639 10538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:20.486819 10538 hybrid_clock.cc:648] HybridClock initialized: now 1786246400486797 us; error 38 us; skew 500 ppm
Aug 09 03:33:20 dist-test-slave-gkw7 krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246400, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:33:20.489526 10538 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:33:20.490131 10538 webserver.cc:460] Webserver started at http://127.27.40.190:41903/ using document root <none> and password file <none>
I20260809 03:33:20.490413 10538 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:20.490494 10538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:20.490680 10538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:33:20.491892 10538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "2e16d230a5804e03aece322405f18e56"
format_stamp: "Formatted at 2026-08-09 03:33:20 on dist-test-slave-gkw7"
server_key: "83a1a421128f1ecd15c8153eec6457e2"
server_key_iv: "98b4878f06c7bbaae5419ffafab9f3e1"
server_key_version: "encryptionkey@0"
I20260809 03:33:20.492323 10538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "2e16d230a5804e03aece322405f18e56"
format_stamp: "Formatted at 2026-08-09 03:33:20 on dist-test-slave-gkw7"
server_key: "83a1a421128f1ecd15c8153eec6457e2"
server_key_iv: "98b4878f06c7bbaae5419ffafab9f3e1"
server_key_version: "encryptionkey@0"
I20260809 03:33:20.493974 10538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260809 03:33:20.494853 10559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:20.495064 10538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 03:33:20.495189 10538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "2e16d230a5804e03aece322405f18e56"
format_stamp: "Formatted at 2026-08-09 03:33:20 on dist-test-slave-gkw7"
server_key: "83a1a421128f1ecd15c8153eec6457e2"
server_key_iv: "98b4878f06c7bbaae5419ffafab9f3e1"
server_key_version: "encryptionkey@0"
I20260809 03:33:20.495280 10538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:20.533744 10538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:20.536110 10538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:20.536266 10538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:20.540971 10538 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:40663
I20260809 03:33:20.541030 10633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:40663 every 8 connection(s)
I20260809 03:33:20.541478 10538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:33:20.542601 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 10538
I20260809 03:33:20.542691 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:33:20.542691 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:33:20.542929 27810 external_mini_cluster.cc:1428] Setting key a98b8e0b38a534e73fe23f14c64e7dc8
I20260809 03:33:20.546300 10634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56: Bootstrap starting.
I20260809 03:33:20.547003 10634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56: Neither blocks nor log segments found. Creating new log.
I20260809 03:33:20.547328 10634 log.cc:824] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56: Log is configured to *not* fsync() on all Append() calls
I20260809 03:33:20.548030 10634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56: No bootstrap required, opened a new log
I20260809 03:33:20.550271 10634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e16d230a5804e03aece322405f18e56" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40663 } }
I20260809 03:33:20.550453 10634 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:33:20.550501 10634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e16d230a5804e03aece322405f18e56, State: Initialized, Role: FOLLOWER
Aug 09 03:33:20 dist-test-slave-gkw7 krb5kdc[10516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:33:20.550642 10634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e16d230a5804e03aece322405f18e56" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40663 } }
I20260809 03:33:20.550733 10634 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:33:20.550782 10634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:33:20.550835 10634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:33:20.551544 10634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e16d230a5804e03aece322405f18e56" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40663 } }
I20260809 03:33:20.551661 10634 leader_election.cc:302] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e16d230a5804e03aece322405f18e56; no voters:
I20260809 03:33:20.551852 10634 leader_election.cc:288] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:33:20.551911 10640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:33:20.552132 10640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [term 1 LEADER]: Becoming Leader. State: Replica: 2e16d230a5804e03aece322405f18e56, State: Running, Role: LEADER
I20260809 03:33:20.552197 10634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:33:20.552232 10640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e16d230a5804e03aece322405f18e56" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40663 } }
I20260809 03:33:20.552599 10641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e16d230a5804e03aece322405f18e56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e16d230a5804e03aece322405f18e56" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40663 } } }
I20260809 03:33:20.552701 10641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [sys.catalog]: This master's current role is: LEADER
I20260809 03:33:20.552599 10643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e16d230a5804e03aece322405f18e56. Latest consensus state: current_term: 1 leader_uuid: "2e16d230a5804e03aece322405f18e56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e16d230a5804e03aece322405f18e56" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40663 } } }
I20260809 03:33:20.552892 10643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56 [sys.catalog]: This master's current role is: LEADER
W20260809 03:33:20.554316 10665 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:33:20.554378 10665 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:33:20.554471 10650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:33:20.554639 10650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:33:20.555773 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:20.543575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52946 (local address 127.27.40.190:40663)
0809 03:33:20.543800 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:20.543812 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:20.543847 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:33:20.544346 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:20.544355 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:20.544422 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:20.544580 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:20.545227 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:20.546667 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:20.547713 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:20.548242 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:20.548443 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:20.551256 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:20.551260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:20.551260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:33:20.551269 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:20.554084 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:20.554488 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:20.554492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:20.554492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:20.554617 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:20.554867 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:20.554868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:20.554869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:20.555120 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:20.555267 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:20.555386 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 03:33:20.555528 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":106,"threads_started":1}
I20260809 03:33:20.558009 10650 catalog_manager.cc:1347] Generated new cluster ID: cc2ec15593584959b4c4d7ca54be9413
I20260809 03:33:20.558107 10650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:33:20.583698 10650 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:33:20.584475 10650 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:33:20.589244 10650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56: Generated new TSK 0
I20260809 03:33:20.589430 10650 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:33:20.618474 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40663
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:43145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:33:20.702703 10673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:20.702914 10673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:20.702939 10673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:20.704078 10673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:20.704123 10673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:20.704149 10673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:33:20.705667 10673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:43145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:20.705920 10673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:20.706174 10673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:20.711314 10681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:20.711373 10684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:20.711314 10682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:20.711513 10673 server_base.cc:1034] running on GCE node
I20260809 03:33:20.711709 10673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:20.711911 10673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:20.713047 10673 hybrid_clock.cc:648] HybridClock initialized: now 1786246400713034 us; error 26 us; skew 500 ppm
Aug 09 03:33:20 dist-test-slave-gkw7 krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246400, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:33:20.720209 10673 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:33:20.720727 10673 webserver.cc:460] Webserver started at http://127.27.40.129:42731/ using document root <none> and password file <none>
I20260809 03:33:20.720968 10673 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:20.721038 10673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:20.721199 10673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:33:20.722383 10673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "6be80d0a34c44f25ae48c73c9ef5771b"
format_stamp: "Formatted at 2026-08-09 03:33:20 on dist-test-slave-gkw7"
server_key: "8297bbd746cfeba701fed11de24fe156"
server_key_iv: "a7e13fe9438755cfb2442080c3e7c00d"
server_key_version: "encryptionkey@0"
I20260809 03:33:20.722759 10673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "6be80d0a34c44f25ae48c73c9ef5771b"
format_stamp: "Formatted at 2026-08-09 03:33:20 on dist-test-slave-gkw7"
server_key: "8297bbd746cfeba701fed11de24fe156"
server_key_iv: "a7e13fe9438755cfb2442080c3e7c00d"
server_key_version: "encryptionkey@0"
I20260809 03:33:20.724161 10673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:33:20.725028 10695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:20.725234 10673 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:33:20.725344 10673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "6be80d0a34c44f25ae48c73c9ef5771b"
format_stamp: "Formatted at 2026-08-09 03:33:20 on dist-test-slave-gkw7"
server_key: "8297bbd746cfeba701fed11de24fe156"
server_key_iv: "a7e13fe9438755cfb2442080c3e7c00d"
server_key_version: "encryptionkey@0"
I20260809 03:33:20.725422 10673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:20.737663 10673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:20.740087 10673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:20.740262 10673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:20.740540 10673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:33:20.740899 10673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:33:20.740955 10673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:20.740998 10673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:33:20.741027 10673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:20.747081 10673 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:38579
I20260809 03:33:20.747131 10844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:38579 every 8 connection(s)
I20260809 03:33:20.747491 10673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
Aug 09 03:33:20 dist-test-slave-gkw7 krb5kdc[10516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246400, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:33:20.754377 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 10673
I20260809 03:33:20.754485 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:33:20.754724 27810 external_mini_cluster.cc:1428] Setting key a8bd91fd6ce5c18d2bd4fb37c865cb7c
I20260809 03:33:20.757581 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:20.748423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:37511 (local address 127.27.40.190:40663)
0809 03:33:20.748501 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:20.748503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:20.748640 (+ 137us) server_negotiation.cc:407] Connection header received
0809 03:33:20.749632 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:20.749633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:20.749645 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:20.749702 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:20.750309 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:20.750811 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:20.751768 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:20.751973 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:20.752025 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:20.754652 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:20.754654 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:20.754654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:33:20.754661 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:20.756132 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:20.756568 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:20.756569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:20.756570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:20.756636 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:20.756892 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:20.756893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:20.756893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:20.757017 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:20.757093 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:20.757373 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 03:33:20.757466 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:33:20.757783 10847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:20.748332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40663 (local address 127.27.40.129:37511)
0809 03:33:20.748502 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:20.748547 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:20.749478 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:20.749760 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:20.749763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:20.750021 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:20.750221 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:20.750225 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:20.750892 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:20.750892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:20.751690 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:20.751692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:20.752048 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:20.752049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:20.752237 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:20.752784 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:20.752794 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:20.754549 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:33:20.756239 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:33:20.756246 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:33:20.756248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:33:20.756478 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:33:20.756707 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:33:20.756707 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:33:20.756708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:33:20.756819 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:33:20.757084 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:33:20.757091 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:33:20.757235 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 03:33:20.757349 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 03:33:20.757509 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260809 03:33:20.758302 10845 heartbeater.cc:347] Connected to a master server at 127.27.40.190:40663
I20260809 03:33:20.758386 10845 heartbeater.cc:464] Registering TS with master...
I20260809 03:33:20.758585 10845 heartbeater.cc:511] Master 127.27.40.190:40663 requested a full tablet report, sending...
I20260809 03:33:20.759116 10584 ts_manager.cc:194] Registered new tserver with Master: 6be80d0a34c44f25ae48c73c9ef5771b (127.27.40.129:38579)
I20260809 03:33:20.759752 10584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:37511
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:33:20.784123 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40663
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:43145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:33:20.870594 10850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:20.870930 10850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:20.870976 10850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:20.872174 10850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:20.872251 10850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:20.872301 10850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:33:20.873924 10850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:43145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:20.874225 10850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:20.874473 10850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:20.879766 10857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:20.879791 10856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:20.879778 10861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:20.880131 10850 server_base.cc:1034] running on GCE node
I20260809 03:33:20.880345 10850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:20.880607 10850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:20.881814 10850 hybrid_clock.cc:648] HybridClock initialized: now 1786246400881806 us; error 55 us; skew 500 ppm
Aug 09 03:33:20 dist-test-slave-gkw7 krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246400, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:33:20.884361 10850 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:33:20.884868 10850 webserver.cc:460] Webserver started at http://127.27.40.130:34383/ using document root <none> and password file <none>
I20260809 03:33:20.885130 10850 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:20.885206 10850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:20.885360 10850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:33:20.886475 10850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "b0f64c1109694418b78c6f0cd00c4a96"
format_stamp: "Formatted at 2026-08-09 03:33:20 on dist-test-slave-gkw7"
server_key: "15c3dfb94b9d568676c6c1770d250653"
server_key_iv: "4079f5254da7002dc157ac263a8664c0"
server_key_version: "encryptionkey@0"
I20260809 03:33:20.886844 10850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "b0f64c1109694418b78c6f0cd00c4a96"
format_stamp: "Formatted at 2026-08-09 03:33:20 on dist-test-slave-gkw7"
server_key: "15c3dfb94b9d568676c6c1770d250653"
server_key_iv: "4079f5254da7002dc157ac263a8664c0"
server_key_version: "encryptionkey@0"
I20260809 03:33:20.888203 10850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:33:20.888989 10874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:20.889227 10850 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:33:20.889346 10850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "b0f64c1109694418b78c6f0cd00c4a96"
format_stamp: "Formatted at 2026-08-09 03:33:20 on dist-test-slave-gkw7"
server_key: "15c3dfb94b9d568676c6c1770d250653"
server_key_iv: "4079f5254da7002dc157ac263a8664c0"
server_key_version: "encryptionkey@0"
I20260809 03:33:20.889454 10850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:20.916314 10850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:20.918613 10850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:20.918769 10850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:20.919032 10850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:33:20.919381 10850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:33:20.919416 10850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:20.919440 10850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:33:20.919482 10850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:20.925714 10850 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:45381
I20260809 03:33:20.925767 11031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:45381 every 8 connection(s)
I20260809 03:33:20.926097 10850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:33:20.930090 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 10850
I20260809 03:33:20.930181 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:33:20.930397 27810 external_mini_cluster.cc:1428] Setting key 3fe9f59361b77cac5ceceb5d270f2c79
Aug 09 03:33:20 dist-test-slave-gkw7 krb5kdc[10516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246400, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:33:20.936326 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:20.926819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:55661 (local address 127.27.40.190:40663)
0809 03:33:20.926892 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:20.926894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:20.927112 (+ 218us) server_negotiation.cc:407] Connection header received
0809 03:33:20.927625 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:20.927626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:20.927638 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:20.927692 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:20.928737 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:20.929324 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:20.930359 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:20.930643 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:20.930706 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:20.933225 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:20.933227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:20.933227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:33:20.933233 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:20.934652 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:20.935171 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:20.935172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:20.935172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:20.935225 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:20.935568 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:20.935570 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:20.935570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:20.935752 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:20.935831 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:20.936185 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 03:33:20.936261 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 03:33:20.936635 11035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:20.926801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40663 (local address 127.27.40.130:55661)
0809 03:33:20.926977 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:20.927015 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:20.927537 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:20.927770 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:20.927773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:20.928444 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:20.928649 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:20.928654 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:20.929402 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:20.929403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:20.930281 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:20.930284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:20.930832 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:20.930833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:20.930974 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:20.931589 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:20.931597 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:20.933117 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:33:20.934738 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:33:20.934745 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:33:20.934747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:33:20.935075 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:33:20.935297 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:33:20.935298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:33:20.935298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:33:20.935472 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:33:20.935846 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:33:20.935854 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:33:20.936035 (+ 181us) client_negotiation.cc:769] Sending connection context
0809 03:33:20.936171 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 03:33:20.936357 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":91,"threads_started":1}
I20260809 03:33:20.937274 11033 heartbeater.cc:347] Connected to a master server at 127.27.40.190:40663
I20260809 03:33:20.937391 11033 heartbeater.cc:464] Registering TS with master...
I20260809 03:33:20.937638 11033 heartbeater.cc:511] Master 127.27.40.190:40663 requested a full tablet report, sending...
I20260809 03:33:20.938114 10584 ts_manager.cc:194] Registered new tserver with Master: b0f64c1109694418b78c6f0cd00c4a96 (127.27.40.130:45381)
I20260809 03:33:20.938511 10584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:55661
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:33:20.961316 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40663
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:43145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:33:21.049259 11041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:33:21.049467 11041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:33:21.049494 11041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:33:21.050637 11041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:33:21.050678 11041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:33:21.050704 11041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:33:21.052172 11041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:43145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:33:21.052390 11041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:33:21.052572 11041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:33:21.058245 11048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:33:21.058216 11052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:21.058290 11041 server_base.cc:1034] running on GCE node
W20260809 03:33:21.058230 11047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:33:21.058698 11041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:33:21.058984 11041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:33:21.060173 11041 hybrid_clock.cc:648] HybridClock initialized: now 1786246401060154 us; error 30 us; skew 500 ppm
Aug 09 03:33:21 dist-test-slave-gkw7 krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246401, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:33:21.062868 11041 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:33:21.063349 11041 webserver.cc:460] Webserver started at http://127.27.40.131:45131/ using document root <none> and password file <none>
I20260809 03:33:21.063578 11041 fs_manager.cc:359] Metadata directory not provided
I20260809 03:33:21.063656 11041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:33:21.063817 11041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:33:21.064862 11041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "f1f382254597477683ef20824e734447"
format_stamp: "Formatted at 2026-08-09 03:33:21 on dist-test-slave-gkw7"
server_key: "3b66907434b308f765c9ba959db03a52"
server_key_iv: "836c03a64c5705039538fa66a9ef493f"
server_key_version: "encryptionkey@0"
I20260809 03:33:21.065234 11041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "f1f382254597477683ef20824e734447"
format_stamp: "Formatted at 2026-08-09 03:33:21 on dist-test-slave-gkw7"
server_key: "3b66907434b308f765c9ba959db03a52"
server_key_iv: "836c03a64c5705039538fa66a9ef493f"
server_key_version: "encryptionkey@0"
I20260809 03:33:21.066700 11041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:33:21.067507 11064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:21.067698 11041 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 03:33:21.067806 11041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "f1f382254597477683ef20824e734447"
format_stamp: "Formatted at 2026-08-09 03:33:21 on dist-test-slave-gkw7"
server_key: "3b66907434b308f765c9ba959db03a52"
server_key_iv: "836c03a64c5705039538fa66a9ef493f"
server_key_version: "encryptionkey@0"
I20260809 03:33:21.067883 11041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:33:21.077741 11041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:33:21.079818 11041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:33:21.079983 11041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:33:21.080233 11041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:33:21.080551 11041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:33:21.080608 11041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:21.080680 11041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:33:21.080713 11041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:33:21.086779 11041 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:46179
I20260809 03:33:21.086824 11227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:46179 every 8 connection(s)
I20260809 03:33:21.087158 11041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
Aug 09 03:33:21 dist-test-slave-gkw7 krb5kdc[10516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246401, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:33:21.097080 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:21.087939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:33099 (local address 127.27.40.190:40663)
0809 03:33:21.088012 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:21.088015 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:21.088230 (+ 215us) server_negotiation.cc:407] Connection header received
0809 03:33:21.088716 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:21.088717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:21.088728 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:21.088781 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:21.089846 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.090350 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.091445 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.091638 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.091692 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:21.094331 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:21.094333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:21.094334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:33:21.094342 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:21.095683 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:21.096101 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:21.096102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:21.096103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:21.096163 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:21.096410 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:21.096411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:21.096412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:21.096531 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:21.096606 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:21.096891 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 03:33:21.096987 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 03:33:21.097294 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:21.087895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40663 (local address 127.27.40.131:33099)
0809 03:33:21.088108 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:21.088147 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:21.088621 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:21.088835 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:21.088839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:21.089500 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:33:21.089751 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:21.089756 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.090436 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.090437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:21.091370 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:21.091372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.091736 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.091738 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:21.091864 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:21.092594 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:33:21.092603 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:33:21.094229 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:33:21.095780 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:33:21.095787 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:33:21.095789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:33:21.096014 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:33:21.096232 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:33:21.096233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:33:21.096233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:33:21.096341 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:33:21.096608 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:33:21.096616 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:33:21.096755 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 03:33:21.096866 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 03:33:21.097020 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":106,"threads_started":1}
I20260809 03:33:21.097494 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 11041
I20260809 03:33:21.097616 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:33:21.097810 27810 external_mini_cluster.cc:1428] Setting key 114cba5e1e9922dd4fe390bfb79a1078
I20260809 03:33:21.097796 11228 heartbeater.cc:347] Connected to a master server at 127.27.40.190:40663
I20260809 03:33:21.097891 11228 heartbeater.cc:464] Registering TS with master...
I20260809 03:33:21.098095 11228 heartbeater.cc:511] Master 127.27.40.190:40663 requested a full tablet report, sending...
I20260809 03:33:21.098569 10582 ts_manager.cc:194] Registered new tserver with Master: f1f382254597477683ef20824e734447 (127.27.40.131:46179)
I20260809 03:33:21.098986 10582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:33099
I20260809 03:33:21.109781 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:33:21.117318 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:21.111201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52962 (local address 127.27.40.190:40663)
0809 03:33:21.111290 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:21.111291 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:21.111339 (+ 48us) server_negotiation.cc:407] Connection header received
0809 03:33:21.111379 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:21.111380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:21.111389 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:21.111449 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:33:21.111988 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.112513 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.113223 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.113403 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.113454 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:21.114703 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:33:21.114704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:33:21.114705 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:33:21.114711 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:33:21.116210 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:21.116534 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:21.116535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:21.116535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:21.116591 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:33:21.116766 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:33:21.116767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:33:21.116767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:33:21.116916 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:33:21.116996 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:21.117163 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 03:33:21.117252 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:33:21.119208 10582 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:33:21.120746 10582 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:33:21.127621 11241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:21.123206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:46179 (local address 127.0.0.1:52086)
0809 03:33:21.123904 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:21.123913 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:21.123965 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:21.124239 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:21.124240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:21.124242 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:33:21.124287 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:21.124289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.125188 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.125189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:21.126506 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:21.126509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.127253 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.127254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:21.127335 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:21.127350 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 03:33:21.127465 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 03:33:21.127529 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":534,"thread_start_us":103,"threads_started":1}
I20260809 03:33:21.127802 11242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:21.123206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:45381 (local address 127.0.0.1:37550)
0809 03:33:21.123366 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:21.123405 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:21.123503 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:21.123950 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:21.123954 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:21.123957 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:33:21.124039 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:21.124041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.125433 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.125434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:21.126581 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:21.126583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.127520 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.127521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:21.127628 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:21.127632 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:33:21.127667 (+ 35us) client_negotiation.cc:240] Negotiation successful
0809 03:33:21.127706 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":49,"thread_start_us":75,"threads_started":1}
I20260809 03:33:21.128077 11243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:21.123356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60050 (local address 127.27.40.129:38579)
0809 03:33:21.123534 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:21.123539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:21.123981 (+ 442us) server_negotiation.cc:407] Connection header received
0809 03:33:21.124108 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:21.124111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:21.124130 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:21.124195 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:33:21.124454 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.125507 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.126704 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.127468 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.127532 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:21.127592 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:21.127873 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 03:33:21.127927 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260809 03:33:21.128170 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:21.123500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37550 (local address 127.27.40.130:45381)
0809 03:33:21.123713 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:21.123717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:21.123728 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:33:21.123772 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:21.123775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:21.123793 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:21.124121 (+ 328us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:33:21.124199 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.125326 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.126650 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.127374 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.127923 (+ 549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:21.127986 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:21.128011 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 03:33:21.128079 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":106,"threads_started":1}
I20260809 03:33:21.128288 11248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:21.123228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:38579 (local address 127.0.0.1:60050)
0809 03:33:21.123911 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:21.123922 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:21.124012 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:21.124289 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:21.124289 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:21.124291 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:33:21.124362 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:21.124364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.125630 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.125631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:21.126591 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:21.126594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.127654 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.127655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:21.127767 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:21.127771 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:33:21.128166 (+ 395us) client_negotiation.cc:240] Negotiation successful
0809 03:33:21.128216 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":1,"thread_start_us":81,"threads_started":1}
I20260809 03:33:21.128657 11246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:21.123564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52086 (local address 127.27.40.131:46179)
0809 03:33:21.123779 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:21.123783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:21.124021 (+ 238us) server_negotiation.cc:407] Connection header received
0809 03:33:21.124077 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:21.124079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:21.124097 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:21.124170 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:33:21.124370 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.125077 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.126603 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.127166 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.127254 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:21.127317 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:21.127520 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 03:33:21.127595 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":119,"threads_started":1}
I20260809 03:33:21.129021 10945 tablet_service.cc:1467] Processing CreateTablet for tablet 0ca8422562f0499dabf8115702580dcc (DEFAULT_TABLE table=test-table [id=89402c9e901148ad905f65c58a2d569f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:33:21.129032 10756 tablet_service.cc:1467] Processing CreateTablet for tablet 0ca8422562f0499dabf8115702580dcc (DEFAULT_TABLE table=test-table [id=89402c9e901148ad905f65c58a2d569f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:33:21.129230 11121 tablet_service.cc:1467] Processing CreateTablet for tablet 0ca8422562f0499dabf8115702580dcc (DEFAULT_TABLE table=test-table [id=89402c9e901148ad905f65c58a2d569f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:33:21.129407 10756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ca8422562f0499dabf8115702580dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:33:21.129419 10945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ca8422562f0499dabf8115702580dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:33:21.129565 11121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ca8422562f0499dabf8115702580dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:33:21.132382 11250 tablet_bootstrap.cc:492] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96: Bootstrap starting.
I20260809 03:33:21.133193 11250 tablet_bootstrap.cc:654] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96: Neither blocks nor log segments found. Creating new log.
I20260809 03:33:21.133431 11251 tablet_bootstrap.cc:492] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447: Bootstrap starting.
I20260809 03:33:21.134495 11250 log.cc:824] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96: Log is configured to *not* fsync() on all Append() calls
I20260809 03:33:21.134550 11251 tablet_bootstrap.cc:654] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447: Neither blocks nor log segments found. Creating new log.
I20260809 03:33:21.135216 11251 log.cc:824] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447: Log is configured to *not* fsync() on all Append() calls
I20260809 03:33:21.135483 11250 tablet_bootstrap.cc:492] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96: No bootstrap required, opened a new log
I20260809 03:33:21.135561 11250 ts_tablet_manager.cc:1397] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:33:21.135597 11252 tablet_bootstrap.cc:492] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b: Bootstrap starting.
I20260809 03:33:21.135967 11251 tablet_bootstrap.cc:492] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447: No bootstrap required, opened a new log
I20260809 03:33:21.136053 11251 ts_tablet_manager.cc:1397] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:33:21.136659 11252 tablet_bootstrap.cc:654] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b: Neither blocks nor log segments found. Creating new log.
I20260809 03:33:21.137558 11252 log.cc:824] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:33:21.138528 11251 raft_consensus.cc:348] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f64c1109694418b78c6f0cd00c4a96" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45381 } } peers { permanent_uuid: "f1f382254597477683ef20824e734447" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46179 } } peers { permanent_uuid: "6be80d0a34c44f25ae48c73c9ef5771b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38579 } }
I20260809 03:33:21.138684 11251 raft_consensus.cc:374] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:33:21.138728 11251 raft_consensus.cc:729] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1f382254597477683ef20824e734447, State: Initialized, Role: FOLLOWER
I20260809 03:33:21.138765 11252 tablet_bootstrap.cc:492] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b: No bootstrap required, opened a new log
I20260809 03:33:21.138710 11250 raft_consensus.cc:348] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f64c1109694418b78c6f0cd00c4a96" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45381 } } peers { permanent_uuid: "f1f382254597477683ef20824e734447" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46179 } } peers { permanent_uuid: "6be80d0a34c44f25ae48c73c9ef5771b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38579 } }
I20260809 03:33:21.138831 11252 ts_tablet_manager.cc:1397] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:33:21.138875 11250 raft_consensus.cc:374] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:33:21.138909 11250 raft_consensus.cc:729] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0f64c1109694418b78c6f0cd00c4a96, State: Initialized, Role: FOLLOWER
I20260809 03:33:21.138895 11251 consensus_queue.cc:260] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f64c1109694418b78c6f0cd00c4a96" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45381 } } peers { permanent_uuid: "f1f382254597477683ef20824e734447" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46179 } } peers { permanent_uuid: "6be80d0a34c44f25ae48c73c9ef5771b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38579 } }
I20260809 03:33:21.139089 11250 consensus_queue.cc:260] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f64c1109694418b78c6f0cd00c4a96" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45381 } } peers { permanent_uuid: "f1f382254597477683ef20824e734447" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46179 } } peers { permanent_uuid: "6be80d0a34c44f25ae48c73c9ef5771b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38579 } }
I20260809 03:33:21.139387 11250 ts_tablet_manager.cc:1428] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 03:33:21.139389 11251 ts_tablet_manager.cc:1428] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:33:21.139444 11228 heartbeater.cc:503] Master 127.27.40.190:40663 was elected leader, sending a full tablet report...
I20260809 03:33:21.139544 11033 heartbeater.cc:503] Master 127.27.40.190:40663 was elected leader, sending a full tablet report...
I20260809 03:33:21.141825 11252 raft_consensus.cc:348] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f64c1109694418b78c6f0cd00c4a96" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45381 } } peers { permanent_uuid: "f1f382254597477683ef20824e734447" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46179 } } peers { permanent_uuid: "6be80d0a34c44f25ae48c73c9ef5771b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38579 } }
I20260809 03:33:21.141963 11252 raft_consensus.cc:374] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:33:21.141984 11252 raft_consensus.cc:729] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6be80d0a34c44f25ae48c73c9ef5771b, State: Initialized, Role: FOLLOWER
I20260809 03:33:21.142123 11252 consensus_queue.cc:260] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f64c1109694418b78c6f0cd00c4a96" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45381 } } peers { permanent_uuid: "f1f382254597477683ef20824e734447" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46179 } } peers { permanent_uuid: "6be80d0a34c44f25ae48c73c9ef5771b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38579 } }
I20260809 03:33:21.142421 11252 ts_tablet_manager.cc:1428] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260809 03:33:21.142544 10845 heartbeater.cc:503] Master 127.27.40.190:40663 was elected leader, sending a full tablet report...
I20260809 03:33:21.165194 11257 raft_consensus.cc:482] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:33:21.165334 11257 raft_consensus.cc:504] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f64c1109694418b78c6f0cd00c4a96" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45381 } } peers { permanent_uuid: "f1f382254597477683ef20824e734447" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46179 } } peers { permanent_uuid: "6be80d0a34c44f25ae48c73c9ef5771b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38579 } }
I20260809 03:33:21.165922 11257 leader_election.cc:288] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0f64c1109694418b78c6f0cd00c4a96 (127.27.40.130:45381), 6be80d0a34c44f25ae48c73c9ef5771b (127.27.40.129:38579)
I20260809 03:33:21.168974 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:21.166168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:54989 (local address 127.27.40.130:45381)
0809 03:33:21.166219 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:21.166221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:21.166230 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:33:21.166331 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:21.166331 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:21.166340 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:21.166418 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:33:21.166677 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.167165 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.168182 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.168609 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.168700 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:21.168710 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:21.168875 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 03:33:21.168908 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 03:33:21.168974 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:21.166086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:45381 (local address 127.27.40.131:54989)
0809 03:33:21.166149 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:21.166161 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:21.166205 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:21.166425 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:21.166426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:21.166428 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:33:21.166594 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:21.166596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.167251 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.167252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:21.168074 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:21.168076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.168699 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.168700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:21.168790 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:21.168793 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:33:21.168856 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 03:33:21.168899 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 03:33:21.169116 11243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:21.166306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:39937 (local address 127.27.40.129:38579)
0809 03:33:21.166360 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:21.166362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:21.166502 (+ 140us) server_negotiation.cc:407] Connection header received
0809 03:33:21.166564 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:21.166565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:21.166574 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:21.166632 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:33:21.166858 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.167350 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.168302 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.168733 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.168816 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:21.168824 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:21.168976 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 03:33:21.169015 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 03:33:21.169379 10971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ca8422562f0499dabf8115702580dcc" candidate_uuid: "f1f382254597477683ef20824e734447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0f64c1109694418b78c6f0cd00c4a96" is_pre_election: true
I20260809 03:33:21.169582 10971 raft_consensus.cc:2454] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1f382254597477683ef20824e734447 in term 0.
I20260809 03:33:21.169711 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:21.166128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:38579 (local address 127.27.40.131:39937)
0809 03:33:21.166427 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:21.166435 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 03:33:21.166507 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:33:21.166641 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:33:21.166641 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:33:21.166643 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:33:21.166777 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:21.166779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.167434 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.167435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:21.168211 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:33:21.168213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.168829 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.168830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:33:21.168901 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:21.168904 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:33:21.168970 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 03:33:21.169010 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":125,"threads_started":1}
I20260809 03:33:21.169926 11066 leader_election.cc:302] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0f64c1109694418b78c6f0cd00c4a96, f1f382254597477683ef20824e734447; no voters:
I20260809 03:33:21.170042 10785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ca8422562f0499dabf8115702580dcc" candidate_uuid: "f1f382254597477683ef20824e734447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6be80d0a34c44f25ae48c73c9ef5771b" is_pre_election: true
I20260809 03:33:21.170145 11257 raft_consensus.cc:2781] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:33:21.170231 11257 raft_consensus.cc:482] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:33:21.170248 10785 raft_consensus.cc:2454] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1f382254597477683ef20824e734447 in term 0.
I20260809 03:33:21.170269 11257 raft_consensus.cc:3037] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:33:21.171223 11257 raft_consensus.cc:504] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f64c1109694418b78c6f0cd00c4a96" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45381 } } peers { permanent_uuid: "f1f382254597477683ef20824e734447" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46179 } } peers { permanent_uuid: "6be80d0a34c44f25ae48c73c9ef5771b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38579 } }
I20260809 03:33:21.171401 11257 leader_election.cc:288] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [CANDIDATE]: Term 1 election: Requested vote from peers b0f64c1109694418b78c6f0cd00c4a96 (127.27.40.130:45381), 6be80d0a34c44f25ae48c73c9ef5771b (127.27.40.129:38579)
I20260809 03:33:21.171536 10971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ca8422562f0499dabf8115702580dcc" candidate_uuid: "f1f382254597477683ef20824e734447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0f64c1109694418b78c6f0cd00c4a96"
I20260809 03:33:21.171574 10785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ca8422562f0499dabf8115702580dcc" candidate_uuid: "f1f382254597477683ef20824e734447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6be80d0a34c44f25ae48c73c9ef5771b"
I20260809 03:33:21.171640 10971 raft_consensus.cc:3037] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:33:21.171646 10785 raft_consensus.cc:3037] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:33:21.172482 10785 raft_consensus.cc:2454] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1f382254597477683ef20824e734447 in term 1.
I20260809 03:33:21.172663 11065 leader_election.cc:302] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6be80d0a34c44f25ae48c73c9ef5771b, f1f382254597477683ef20824e734447; no voters:
I20260809 03:33:21.172811 10971 raft_consensus.cc:2454] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1f382254597477683ef20824e734447 in term 1.
I20260809 03:33:21.172930 11257 raft_consensus.cc:2781] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:33:21.173103 11257 raft_consensus.cc:686] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [term 1 LEADER]: Becoming Leader. State: Replica: f1f382254597477683ef20824e734447, State: Running, Role: LEADER
I20260809 03:33:21.173202 11257 consensus_queue.cc:237] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f64c1109694418b78c6f0cd00c4a96" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45381 } } peers { permanent_uuid: "f1f382254597477683ef20824e734447" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46179 } } peers { permanent_uuid: "6be80d0a34c44f25ae48c73c9ef5771b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38579 } }
I20260809 03:33:21.174043 10584 catalog_manager.cc:5495] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1f382254597477683ef20824e734447 (127.27.40.131). New cstate: current_term: 1 leader_uuid: "f1f382254597477683ef20824e734447" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f64c1109694418b78c6f0cd00c4a96" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1f382254597477683ef20824e734447" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 46179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6be80d0a34c44f25ae48c73c9ef5771b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38579 } health_report { overall_health: UNKNOWN } } }
W20260809 03:33:21.176571 11034 tablet.cc:2369] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:33:21.188818 11246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:21.185740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52102 (local address 127.27.40.131:46179)
0809 03:33:21.185804 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 03:33:21.185806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:33:21.185815 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:33:21.185853 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:33:21.185854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:33:21.185875 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:33:21.185961 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:33:21.186132 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.186621 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.187762 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:33:21.188119 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:33:21.188213 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:33:21.188386 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:33:21.188517 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:33:21.188597 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:33:21.188644 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 03:33:21.188665 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 03:33:21.190965 10971 raft_consensus.cc:1261] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96 [term 1 FOLLOWER]: Refusing update from remote peer f1f382254597477683ef20824e734447: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:33:21.190966 10785 raft_consensus.cc:1261] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b [term 1 FOLLOWER]: Refusing update from remote peer f1f382254597477683ef20824e734447: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:33:21.191372 11257 consensus_queue.cc:1035] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6be80d0a34c44f25ae48c73c9ef5771b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:33:21.191561 11265 consensus_queue.cc:1035] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0f64c1109694418b78c6f0cd00c4a96" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:33:21.195098 11272 mvcc.cc:205] Tried to move back new op lower bound from 7316465259275558912 to 7316465259206082560. Current Snapshot: MvccSnapshot[applied={T|T < 7316465259275558912}]
I20260809 03:33:21.197351 10584 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:33:21.197475 10584 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:33:21.198366 10584 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e16d230a5804e03aece322405f18e56: Sending DeleteTablet for 3 replicas of tablet 0ca8422562f0499dabf8115702580dcc
I20260809 03:33:21.198760 10945 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ca8422562f0499dabf8115702580dcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:33:21 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:37550
I20260809 03:33:21.198787 10756 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ca8422562f0499dabf8115702580dcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:33:21 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:60050
I20260809 03:33:21.199028 11121 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ca8422562f0499dabf8115702580dcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:33:21 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:52086
I20260809 03:33:21.199200 11285 tablet_replica.cc:330] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b: stopping tablet replica
I20260809 03:33:21.199345 11285 raft_consensus.cc:2229] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:33:21.199471 11285 raft_consensus.cc:2258] T 0ca8422562f0499dabf8115702580dcc P 6be80d0a34c44f25ae48c73c9ef5771b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:33:21.199680 11288 tablet_replica.cc:330] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447: stopping tablet replica
I20260809 03:33:21.199750 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 10673
I20260809 03:33:21.199857 11288 raft_consensus.cc:2229] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:33:21.200023 11288 raft_consensus.cc:2258] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:33:21.215585 11288 ts_tablet_manager.cc:1905] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:33:21.217016 11283 tablet_replica.cc:330] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96: stopping tablet replica
I20260809 03:33:21.217250 11283 raft_consensus.cc:2229] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:33:21.217449 11283 raft_consensus.cc:2258] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:33:21.217938 11288 ts_tablet_manager.cc:1918] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:33:21.218079 11288 log.cc:1196] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/0ca8422562f0499dabf8115702580dcc
I20260809 03:33:21.218293 11283 ts_tablet_manager.cc:1905] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:33:21.218554 11288 ts_tablet_manager.cc:1939] T 0ca8422562f0499dabf8115702580dcc P f1f382254597477683ef20824e734447: Deleting consensus metadata
I20260809 03:33:21.220575 11283 ts_tablet_manager.cc:1918] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:33:21.220669 11283 log.cc:1196] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/0ca8422562f0499dabf8115702580dcc
I20260809 03:33:21.221022 11283 ts_tablet_manager.cc:1939] T 0ca8422562f0499dabf8115702580dcc P b0f64c1109694418b78c6f0cd00c4a96: Deleting consensus metadata
I20260809 03:33:21.221642 10561 catalog_manager.cc:4841] TS b0f64c1109694418b78c6f0cd00c4a96 (127.27.40.130:45381): tablet 0ca8422562f0499dabf8115702580dcc (table test-table [id=89402c9e901148ad905f65c58a2d569f]) successfully deleted
W20260809 03:33:21.223155 10560 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.40.129:38579 (error 108)
I20260809 03:33:21.223534 10567 catalog_manager.cc:4841] TS f1f382254597477683ef20824e734447 (127.27.40.131:46179): tablet 0ca8422562f0499dabf8115702580dcc (table test-table [id=89402c9e901148ad905f65c58a2d569f]) successfully deleted
I20260809 03:33:21.223896 11248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:33:21.223741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:38579 (local address 127.0.0.1:60052)
0809 03:33:21.223801 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 03:33:21.223852 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:38579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 03:33:21.224011 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 10850
W20260809 03:33:21.224120 10560 catalog_manager.cc:4584] TS 6be80d0a34c44f25ae48c73c9ef5771b (127.27.40.129:38579): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0ca8422562f0499dabf8115702580dcc: Network error: Client connection negotiation failed: client connection to 127.27.40.129:38579: connect: Connection refused (error 111)
I20260809 03:33:21.242182 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 11041
I20260809 03:33:21.251456 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 10538
2026-08-09T03:33:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2265 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:33:21 dist-test-slave-gkw7 krb5kdc[11293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:33:21 dist-test-slave-gkw7 krb5kdc[11293](info): set up 2 sockets
Aug 09 03:33:21 dist-test-slave-gkw7 krb5kdc[11293](info): commencing operation
krb5kdc: starting...
W20260809 03:33:22.541145 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:33:22 dist-test-slave-gkw7 krb5kdc[11293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:33:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:33:22Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.27.40.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.27.40.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.27.40.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/rangeradmin_127.27.40.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.27.40.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/rangeradmin_127.27.40.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.27.40.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.27.40.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.27.40.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/rangerlookup_127.27.40.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.27.40.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/rangerlookup_127.27.40.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.40.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.27.40.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.40.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/HTTP_127.27.40.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.40.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/HTTP_127.27.40.148@KRBTEST.COM.keytab.
I20260809 03:33:22.720594 27810 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskcMSfqS/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/postgres -l logfile start
2026-08-09 03:33:25.353 UTC [11337] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 03:33:25.353 UTC [11337] LOG: listening on IPv4 address "127.27.40.148", port 32963
2026-08-09 03:33:25.356 UTC [11337] LOG: listening on Unix socket "/tmp/.s.PGSQL.32963"
2026-08-09 03:33:25.380 UTC [11347] LOG: database system was shut down at 2026-08-09 03:33:23 UTC
2026-08-09 03:33:25.384 UTC [11337] LOG: database system is ready to accept connections
I20260809 03:33:26.530253 27810 mini_postgres.cc:96] Postgres bound to 32963
2026-08-09 03:33:26.537 UTC [11356] FATAL: database "slave" does not exist
127.27.40.148:32963 - accepting connections
I20260809 03:33:26.538363 27810 mini_ranger.cc:161] Starting Ranger...
I20260809 03:33:26.554071 27810 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 03:33:27.061146 27810 mini_ranger.cc:87] Created ranger Postgres database
I20260809 03:33:27.061259 27810 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-admin
2026-08-09 03:33:27,459 [I] DB FLAVOR :POSTGRES
2026-08-09 03:33:27,460 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 03:33:27,460 [I] Checking connection..
2026-08-09 03:33:27,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 03:33:27,778 [I] Checking connection passed.
2026-08-09 03:33:27,778 [I] --------- Verifying version history table ---------
2026-08-09 03:33:27,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:33:28,058 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 03:33:28,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:33:28,318 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 03:33:28,318 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 03:33:28,318 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 03:33:28.584 UTC [11440] WARNING: there is no transaction in progress
2026-08-09 03:33:28,593 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 03:33:28,593 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:33:28,859 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 03:33:28,860 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 03:33:28,860 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 03:33:29,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 03:33:29,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gkw7.c.gcp-upstream.internal','N') ;"
2026-08-09 03:33:29,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 03:33:29,879 [I] Table x_portal_user does not exist in database ranger
2026-08-09 03:33:29,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 03:33:30,149 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 03:33:30,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 03:33:30,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 03:33:30,711 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 03:33:30,711 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 03:33:31.717 UTC [11597] WARNING: there is no transaction in progress
2026-08-09 03:33:31.732 UTC [11597] WARNING: there is no transaction in progress
2026-08-09 03:33:31.748 UTC [11597] WARNING: there is no transaction in progress
2026-08-09 03:33:31.763 UTC [11597] WARNING: there is no transaction in progress
2026-08-09 03:33:31.779 UTC [11597] WARNING: there is no transaction in progress
2026-08-09 03:33:31.798 UTC [11597] WARNING: there is no transaction in progress
2026-08-09 03:33:31.855 UTC [11597] WARNING: there is no transaction in progress
2026-08-09 03:33:31.861 UTC [11597] WARNING: there is no transaction in progress
2026-08-09 03:33:31.876 UTC [11597] WARNING: there is no transaction in progress
2026-08-09 03:33:31.886 UTC [11597] WARNING: there is no transaction in progress
2026-08-09 03:33:31.897 UTC [11597] WARNING: there is no transaction in progress
2026-08-09 03:33:31.910 UTC [11597] WARNING: there is no transaction in progress
2026-08-09 03:33:31.920 UTC [11597] WARNING: there is no transaction in progress
2026-08-09 03:33:31.926 UTC [11597] WARNING: there is no transaction in progress
2026-08-09 03:33:31.934 UTC [11597] WARNING: there is no transaction in progress
2026-08-09 03:33:31.942 UTC [11597] WARNING: there is no transaction in progress
2026-08-09 03:33:32,584 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 03:33:32,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 03:33:32,841 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 03:33:32,841 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 03:33:33,095 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 03:33:33,096 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 03:33:33,356 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 03:33:33,357 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 03:33:33,608 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 03:33:33,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gkw7.c.gcp-upstream.internal';"
2026-08-09 03:33:34,137 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 03:33:34,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 03:33:34,405 [I] DB_PATCHES have already been applied
I20260809 03:33:34.410928 27810 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-admin:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskcMSfqS/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 03:33:34.411000 27810 mini_ranger.cc:193] Using host: 127.27.40.148
I20260809 03:33:34.414671 27810 mini_ranger.cc:227] Ranger admin URL: http://127.27.40.148:33289
Aug 09, 2026 3:33:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:33:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:33:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 3:33:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 3:33:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 3:33:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 3:33:35 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 03:33:35.857331 27810 mini_ranger.cc:160] Time spent starting Ranger: real 9.319s user 0.005s sys 0.007s
Aug 09, 2026 3:33:55 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 3:33:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 3:33:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 3:33:55 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 3:33:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 3:33:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 3:33:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 3:33:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 3:33:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 3:33:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 3:33:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 3:33:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 3:33:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 3:33:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 3:33:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 3:33:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 3:33:55 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 3:33:56 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 03:33:56.774611 27810 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.27.40.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.27.40.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.27.40.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/rangerkms_127.27.40.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.27.40.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/rangerkms_127.27.40.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.40.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.27.40.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.40.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/HTTP_127.27.40.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.40.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/HTTP_127.27.40.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 03:33:56 dist-test-slave-gkw7 krb5kdc[11293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246436, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 03:33:56.872021 27810 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 03:33:56.882086 27810 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 03:33:57.793696 27810 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 03:33:57.793805 27810 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-kms
I20260809 03:33:57.793825 27810 mini_ranger_kms.cc:227] Using postgres at 127.27.40.148:32963
2026-08-09 03:33:58,395 [I] DB FLAVOR :POSTGRES
2026-08-09 03:33:58,396 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 03:33:58,396 [I] Checking connection
2026-08-09 03:33:58,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 03:33:58,698 [I] connection success
2026-08-09 03:33:58,698 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 03:33:58,699 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 03:33:58,699 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 03:33:58,961 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 03:33:58,962 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 03:33:58,962 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 03:33:58,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:32963/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 03:33:59,221 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 03:33:59.704789 27810 mini_ranger_kms.cc:310] Created kms service
I20260809 03:33:59.938473 27810 mini_ranger_kms.cc:326] Created kudu user
I20260809 03:33:59.983136 27810 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 03:33:59 dist-test-slave-gkw7 krb5kdc[11293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246436, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
I20260809 03:34:00.393601 27810 mini_ranger_kms.cc:384] Added ranger policy
I20260809 03:34:00.393723 27810 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-kms:/tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 03:34:00.393747 27810 mini_ranger_kms.cc:240] Using host: 127.27.40.148
I20260809 03:34:00.397930 27810 mini_ranger_kms.cc:276] Ranger KMS PID: 11865
I20260809 03:34:00.398059 27810 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.27.40.148:37597
Aug 09, 2026 3:34:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:34:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:34:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 3:34:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 3:34:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 3:34:02 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 03:34:03 dist-test-slave-gkw7 krb5kdc[11293](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786246443, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.40.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:03.121219 27810 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.249s user 0.000s sys 0.578s
I20260809 03:34:03.121315 27810 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 03:34:03.121378 27810 mini_ranger_kms.cc:401] 127.27.40.148:37597/kms/v1/keys
Aug 09 03:34:03 dist-test-slave-gkw7 krb5kdc[11293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246443, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.40.148@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
Aug 09 03:34:03 dist-test-slave-gkw7 krb5kdc[11293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246443, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.40.148@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
Aug 09 03:34:06 dist-test-slave-gkw7 krb5kdc[11293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:34:06.372891 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:41723
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:46517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:41723
--ranger_config_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.40.148:37597/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:34:06.471099 11940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:06.471347 11940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:06.471390 11940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:06.472570 11940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:34:06.472642 11940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:06.472672 11940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:06.472707 11940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:34:06.472734 11940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:34:06.474596 11940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:46517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/wal
--ranger_kms_url=127.27.40.148:37597/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:41723
--ranger_config_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:41723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:34:06.474890 11940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:06.475126 11940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:06.481159 11951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:06.481172 11957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:06.481279 11940 server_base.cc:1034] running on GCE node
W20260809 03:34:06.481228 11955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:06.481693 11940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:06.482018 11940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:06.483165 11940 hybrid_clock.cc:648] HybridClock initialized: now 1786246446483160 us; error 37 us; skew 500 ppm
Aug 09 03:34:06 dist-test-slave-gkw7 krb5kdc[11293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246446, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:06.491269 11940 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:34:06.491763 11940 webserver.cc:460] Webserver started at http://127.27.40.190:45111/ using document root <none> and password file <none>
I20260809 03:34:06.492018 11940 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:06.492085 11940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:34:06.492250 11940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:34:06 dist-test-slave-gkw7 krb5kdc[11293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246446, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
I20260809 03:34:06.549182 11940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "da7e3f2652274b74bd567ac2a623e7e0"
format_stamp: "Formatted at 2026-08-09 03:34:06 on dist-test-slave-gkw7"
server_key: "73cbfc0f8e2e3d243f62c763fb4f5a9c"
server_key_iv: "aa4e9b9420e54197cbf5667760a1df9f"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:06.549810 11940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "da7e3f2652274b74bd567ac2a623e7e0"
format_stamp: "Formatted at 2026-08-09 03:34:06 on dist-test-slave-gkw7"
server_key: "73cbfc0f8e2e3d243f62c763fb4f5a9c"
server_key_iv: "aa4e9b9420e54197cbf5667760a1df9f"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:06.551707 11940 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260809 03:34:06.565706 11966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:06.565950 11940 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 03:34:06.566078 11940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "da7e3f2652274b74bd567ac2a623e7e0"
format_stamp: "Formatted at 2026-08-09 03:34:06 on dist-test-slave-gkw7"
server_key: "73cbfc0f8e2e3d243f62c763fb4f5a9c"
server_key_iv: "aa4e9b9420e54197cbf5667760a1df9f"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:06.566169 11940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:06.582134 11940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:06.584178 11940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:06.584369 11940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:06.594503 11940 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:41723
I20260809 03:34:06.594599 12043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:41723 every 8 connection(s)
I20260809 03:34:06.594957 11940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:34:06.595917 12044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:34:06.598917 12044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0: Bootstrap starting.
I20260809 03:34:06.599835 12044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0: Neither blocks nor log segments found. Creating new log.
I20260809 03:34:06.600155 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 11940
I20260809 03:34:06.600272 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:34:06.600459 27810 external_mini_cluster.cc:1428] Setting key 59e1d625a404170e1548ed49d16570b6
I20260809 03:34:06.600453 12044 log.cc:824] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0: Log is configured to *not* fsync() on all Append() calls
I20260809 03:34:06.601454 12044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0: No bootstrap required, opened a new log
I20260809 03:34:06.606282 12044 raft_consensus.cc:348] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da7e3f2652274b74bd567ac2a623e7e0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41723 } }
I20260809 03:34:06.606472 12044 raft_consensus.cc:374] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:34:06.606551 12044 raft_consensus.cc:729] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7e3f2652274b74bd567ac2a623e7e0, State: Initialized, Role: FOLLOWER
I20260809 03:34:06.606745 12044 consensus_queue.cc:260] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da7e3f2652274b74bd567ac2a623e7e0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41723 } }
I20260809 03:34:06.606859 12044 raft_consensus.cc:388] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:34:06.606930 12044 raft_consensus.cc:482] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:34:06.607007 12044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:34:06.608120 12044 raft_consensus.cc:504] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da7e3f2652274b74bd567ac2a623e7e0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41723 } }
I20260809 03:34:06.608291 12044 leader_election.cc:302] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da7e3f2652274b74bd567ac2a623e7e0; no voters:
I20260809 03:34:06.608572 12044 leader_election.cc:288] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:34:06.608685 12049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:34:06.608985 12049 raft_consensus.cc:686] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [term 1 LEADER]: Becoming Leader. State: Replica: da7e3f2652274b74bd567ac2a623e7e0, State: Running, Role: LEADER
I20260809 03:34:06.609143 12044 sys_catalog.cc:564] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:34:06.609179 12049 consensus_queue.cc:237] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da7e3f2652274b74bd567ac2a623e7e0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41723 } }
I20260809 03:34:06.609776 12049 sys_catalog.cc:455] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da7e3f2652274b74bd567ac2a623e7e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da7e3f2652274b74bd567ac2a623e7e0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41723 } } }
I20260809 03:34:06.609992 12049 sys_catalog.cc:458] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [sys.catalog]: This master's current role is: LEADER
Aug 09 03:34:06 dist-test-slave-gkw7 krb5kdc[11293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:34:06.610360 12049 sys_catalog.cc:455] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da7e3f2652274b74bd567ac2a623e7e0. Latest consensus state: current_term: 1 leader_uuid: "da7e3f2652274b74bd567ac2a623e7e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da7e3f2652274b74bd567ac2a623e7e0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41723 } } }
I20260809 03:34:06.610648 12049 sys_catalog.cc:458] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0 [sys.catalog]: This master's current role is: LEADER
I20260809 03:34:06.611018 12061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:34:06.613662 12061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:34:06.614310 12044 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 03:34:06.617515 12048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:06.601074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46842 (local address 127.27.40.190:41723)
0809 03:34:06.601366 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:06.601376 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:06.601458 (+ 82us) server_negotiation.cc:407] Connection header received
0809 03:34:06.602432 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:06.602443 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:06.602547 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:06.602766 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:06.603602 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:06.605643 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:06.606947 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:06.607706 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:06.607826 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:06.610920 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:06.610924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:06.610925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:06.610949 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:06.615493 (+ 4544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:06.615996 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:06.616000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:06.616000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:06.616107 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:06.616381 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:06.616382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:06.616383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:06.616663 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:06.616897 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:06.617029 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 03:34:06.617166 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":122,"threads_started":1}
I20260809 03:34:06.624208 12061 catalog_manager.cc:1347] Generated new cluster ID: fc67d0dd7a164ef0a5c128e4337d4735
I20260809 03:34:06.624286 12061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:34:06.638998 12061 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:34:06.639927 12061 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:34:06.651789 12061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0: Generated new TSK 0
I20260809 03:34:06.652158 12061 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:34:07.472493 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.40.148:37597/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:41723
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:46517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:34:07.576538 12113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:07.576748 12113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:07.576772 12113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:07.578069 12113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:07.578116 12113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:07.578143 12113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:34:07.579667 12113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:46517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.27.40.148:37597/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:41723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:34:07.579905 12113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:07.580175 12113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:07.585920 12119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:07.586092 12120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:07.586170 12123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:07.586752 12113 server_base.cc:1034] running on GCE node
I20260809 03:34:07.586970 12113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:07.587234 12113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:07.588385 12113 hybrid_clock.cc:648] HybridClock initialized: now 1786246447588374 us; error 36 us; skew 500 ppm
Aug 09 03:34:07 dist-test-slave-gkw7 krb5kdc[11293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246447, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:07.591538 12113 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:34:07.592020 12113 webserver.cc:460] Webserver started at http://127.27.40.129:45425/ using document root <none> and password file <none>
I20260809 03:34:07.592260 12113 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:07.592329 12113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:34:07.592486 12113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:34:07 dist-test-slave-gkw7 krb5kdc[11293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246447, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
I20260809 03:34:07.612283 12113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "3363af8add0746eab2d905afed91c8fd"
format_stamp: "Formatted at 2026-08-09 03:34:07 on dist-test-slave-gkw7"
server_key: "7ad8adc89bf12992163ff92bd3e0f336"
server_key_iv: "ea66f41b3de4ab635fd67a302c06b3db"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:07.612977 12113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "3363af8add0746eab2d905afed91c8fd"
format_stamp: "Formatted at 2026-08-09 03:34:07 on dist-test-slave-gkw7"
server_key: "7ad8adc89bf12992163ff92bd3e0f336"
server_key_iv: "ea66f41b3de4ab635fd67a302c06b3db"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:07.615533 12113 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 03:34:07.629488 12137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:07.629789 12113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 03:34:07.629930 12113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "3363af8add0746eab2d905afed91c8fd"
format_stamp: "Formatted at 2026-08-09 03:34:07 on dist-test-slave-gkw7"
server_key: "7ad8adc89bf12992163ff92bd3e0f336"
server_key_iv: "ea66f41b3de4ab635fd67a302c06b3db"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:07.630048 12113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:07.652621 12113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:07.655391 12113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:07.655670 12113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:07.656123 12113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:34:07.656569 12113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:34:07.656633 12113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:07.656733 12113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:34:07.656766 12113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:07.669468 12113 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:37637
I20260809 03:34:07.669718 12301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:37637 every 8 connection(s)
I20260809 03:34:07.669932 12113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
Aug 09 03:34:07 dist-test-slave-gkw7 krb5kdc[11293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246447, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:34:07.679498 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 12113
I20260809 03:34:07.679615 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:34:07.679840 27810 external_mini_cluster.cc:1428] Setting key 50f287e2b1db03b83c15d301f9cad91c
I20260809 03:34:07.683213 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:07.670955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:54183 (local address 127.27.40.190:41723)
0809 03:34:07.671106 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:07.671108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:07.671417 (+ 309us) server_negotiation.cc:407] Connection header received
0809 03:34:07.672456 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:07.672457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:07.672471 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:07.672535 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:07.673474 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:07.674127 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:07.675811 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:07.676013 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:07.676077 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:07.679170 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:07.679172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:07.679172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:07.679179 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:07.681468 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:07.682048 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:07.682050 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:07.682050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:07.682129 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:07.682511 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:07.682512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:07.682513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:07.682647 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:07.682728 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:07.683029 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 03:34:07.683116 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
I20260809 03:34:07.683462 12313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:07.670927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:41723 (local address 127.27.40.129:54183)
0809 03:34:07.671213 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:07.671274 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:07.672327 (+ 1053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:07.672632 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:07.672638 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:07.673005 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:34:07.673377 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:07.673383 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:07.674240 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:07.674241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:07.675718 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:07.675721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:07.676117 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:07.676119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:07.676315 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:07.677036 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:34:07.677046 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:34:07.678976 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:34:07.681585 (+ 2609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:07.681592 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:07.681595 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:07.681935 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:07.682301 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:07.682302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:07.682302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:07.682445 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:07.682729 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:34:07.682737 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:34:07.682876 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 03:34:07.682991 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 03:34:07.683152 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":131,"threads_started":1}
I20260809 03:34:07.684082 12306 heartbeater.cc:347] Connected to a master server at 127.27.40.190:41723
I20260809 03:34:07.684195 12306 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:07.684393 12306 heartbeater.cc:511] Master 127.27.40.190:41723 requested a full tablet report, sending...
I20260809 03:34:07.685031 11993 ts_manager.cc:194] Registered new tserver with Master: 3363af8add0746eab2d905afed91c8fd (127.27.40.129:37637)
I20260809 03:34:07.685976 11993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:54183
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:34:07.713678 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.40.148:37597/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:41723
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:46517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:34:07.855902 12318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:07.856238 12318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:07.856279 12318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:07.858079 12318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:07.858140 12318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:07.858179 12318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:34:07.860633 12318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:46517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.27.40.148:37597/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:41723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:34:07.860921 12318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:07.861291 12318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:07.869458 12328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:07.869751 12327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:07.869971 12332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:07.870985 12318 server_base.cc:1034] running on GCE node
I20260809 03:34:07.871222 12318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:07.871502 12318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:07.872682 12318 hybrid_clock.cc:648] HybridClock initialized: now 1786246447872649 us; error 49 us; skew 500 ppm
Aug 09 03:34:07 dist-test-slave-gkw7 krb5kdc[11293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246447, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:07.876461 12318 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:34:07.877099 12318 webserver.cc:460] Webserver started at http://127.27.40.130:35719/ using document root <none> and password file <none>
I20260809 03:34:07.877444 12318 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:07.877580 12318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:34:07.877751 12318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:34:07 dist-test-slave-gkw7 krb5kdc[11293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246447, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
I20260809 03:34:07.892985 12318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "6ebcd3a95a234d94aa6a621782b80bcb"
format_stamp: "Formatted at 2026-08-09 03:34:07 on dist-test-slave-gkw7"
server_key: "77048eb01aef17e3d82d1c95e592d371"
server_key_iv: "f48f49480be73687ba4b6581d38eafa6"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:07.893565 12318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "6ebcd3a95a234d94aa6a621782b80bcb"
format_stamp: "Formatted at 2026-08-09 03:34:07 on dist-test-slave-gkw7"
server_key: "77048eb01aef17e3d82d1c95e592d371"
server_key_iv: "f48f49480be73687ba4b6581d38eafa6"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:07.895808 12318 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:34:07.913733 12343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:07.913950 12318 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260809 03:34:07.914028 12318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "6ebcd3a95a234d94aa6a621782b80bcb"
format_stamp: "Formatted at 2026-08-09 03:34:07 on dist-test-slave-gkw7"
server_key: "77048eb01aef17e3d82d1c95e592d371"
server_key_iv: "f48f49480be73687ba4b6581d38eafa6"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:07.914120 12318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 09 03:34:07 dist-test-slave-gkw7 krb5kdc[11293](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786246447, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:07.933174 12318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:07.935771 12318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:07.936028 12318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:07.936386 12318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:34:07.936808 12318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:34:07.936889 12318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:07.936950 12318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:34:07.936995 12318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:07.951581 12318 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:43345
I20260809 03:34:07.951891 12509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:43345 every 8 connection(s)
I20260809 03:34:07.952221 12318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:34:07.961012 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 12318
Aug 09 03:34:07 dist-test-slave-gkw7 krb5kdc[11293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246447, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:34:07.961144 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:34:07.961380 27810 external_mini_cluster.cc:1428] Setting key 5d2ea49a30c53dc9f20736bfcfb8f95b
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
I20260809 03:34:07.980530 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:07.953668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:56727 (local address 127.27.40.190:41723)
0809 03:34:07.953719 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:07.953721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:07.954246 (+ 525us) server_negotiation.cc:407] Connection header received
0809 03:34:07.955298 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:07.955300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:07.955319 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:07.955351 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:07.956513 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:07.957252 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:07.958663 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:07.958887 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:07.958962 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:07.974944 (+ 15982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:07.974947 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:07.974947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:07.974956 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:07.976856 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:07.977467 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:07.977468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:07.977468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:07.977559 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:07.977867 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:07.977868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:07.977869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:07.978075 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:07.978170 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:07.980260 (+ 2090us) server_negotiation.cc:299] Negotiation successful
0809 03:34:07.980376 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 03:34:07.980721 12512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:07.953671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:41723 (local address 127.27.40.130:56727)
0809 03:34:07.954094 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:07.954139 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:07.955163 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:07.955536 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:07.955540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:07.955994 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:34:07.956377 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:07.956383 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:07.957381 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:07.957382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:07.958559 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:07.958561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:07.958982 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:07.958983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:07.959108 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:07.959888 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:34:07.959896 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:34:07.974771 (+ 14875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:34:07.976993 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:07.977005 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:07.977008 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:07.977367 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:07.977638 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:07.977639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:07.977639 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:07.977784 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:07.978207 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:34:07.978214 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:34:07.978392 (+ 178us) client_negotiation.cc:769] Sending connection context
0809 03:34:07.980211 (+ 1819us) client_negotiation.cc:240] Negotiation successful
0809 03:34:07.980420 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":169,"threads_started":1}
I20260809 03:34:07.981421 12510 heartbeater.cc:347] Connected to a master server at 127.27.40.190:41723
I20260809 03:34:07.981545 12510 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:07.981756 12510 heartbeater.cc:511] Master 127.27.40.190:41723 requested a full tablet report, sending...
I20260809 03:34:07.982240 11993 ts_manager.cc:194] Registered new tserver with Master: 6ebcd3a95a234d94aa6a621782b80bcb (127.27.40.130:43345)
I20260809 03:34:07.982790 11993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:56727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:34:08.000957 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.40.148:37597/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:41723
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:46517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:34:08.123575 12519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:08.123947 12519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:08.124029 12519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:08.125793 12519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:08.125883 12519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:08.125985 12519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:34:08.127861 12519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:46517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.27.40.148:37597/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:41723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:34:08.128278 12519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:08.128580 12519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:08.134608 12532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:08.134807 12529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:08.134608 12528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:08.136281 12519 server_base.cc:1034] running on GCE node
I20260809 03:34:08.136507 12519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:08.136795 12519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:08.137964 12519 hybrid_clock.cc:648] HybridClock initialized: now 1786246448137943 us; error 43 us; skew 500 ppm
Aug 09 03:34:08 dist-test-slave-gkw7 krb5kdc[11293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246448, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:08.141526 12519 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:34:08.142094 12519 webserver.cc:460] Webserver started at http://127.27.40.131:43095/ using document root <none> and password file <none>
I20260809 03:34:08.142413 12519 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:08.142495 12519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:34:08.142822 12519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:34:08 dist-test-slave-gkw7 krb5kdc[11293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246448, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
I20260809 03:34:08.162997 12519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "2b166b2070fb428bbde43505c54cc594"
format_stamp: "Formatted at 2026-08-09 03:34:08 on dist-test-slave-gkw7"
server_key: "2469b56c9f9bee43fa71799bf380670e"
server_key_iv: "68cd44a7995a8e51647aa149a9b30fee"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:08.163586 12519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "2b166b2070fb428bbde43505c54cc594"
format_stamp: "Formatted at 2026-08-09 03:34:08 on dist-test-slave-gkw7"
server_key: "2469b56c9f9bee43fa71799bf380670e"
server_key_iv: "68cd44a7995a8e51647aa149a9b30fee"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:08.173089 12519 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.001s sys 0.001s
I20260809 03:34:08.189770 12540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:08.190054 12519 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260809 03:34:08.190229 12519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "2b166b2070fb428bbde43505c54cc594"
format_stamp: "Formatted at 2026-08-09 03:34:08 on dist-test-slave-gkw7"
server_key: "2469b56c9f9bee43fa71799bf380670e"
server_key_iv: "68cd44a7995a8e51647aa149a9b30fee"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:08.190423 12519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:08.224781 12519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:08.227064 12519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:08.227242 12519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:08.227548 12519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:34:08.228003 12519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:34:08.228058 12519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:08.228144 12519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:34:08.228175 12519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:08.244477 12519 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:45765
I20260809 03:34:08.244597 12703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:45765 every 8 connection(s)
I20260809 03:34:08.245042 12519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:34:08.251128 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 12519
I20260809 03:34:08.251291 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:34:08.251521 27810 external_mini_cluster.cc:1428] Setting key 0e439f46b5b1c469d05b53b1d9aa4d24
Aug 09 03:34:08 dist-test-slave-gkw7 krb5kdc[11293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246448, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:34:08.258636 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:08.246422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:60921 (local address 127.27.40.190:41723)
0809 03:34:08.246474 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:08.246477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:08.246959 (+ 482us) server_negotiation.cc:407] Connection header received
0809 03:34:08.247595 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:08.247596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:08.247610 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:08.247680 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:08.248361 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.248872 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.249980 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.250201 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.250279 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:08.254783 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:08.254785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:08.254786 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:08.254794 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:08.256768 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:08.257358 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:08.257359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:08.257359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:08.257427 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:08.257791 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:08.257792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:08.257793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:08.257938 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:08.257986 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:08.258434 (+ 448us) server_negotiation.cc:299] Negotiation successful
0809 03:34:08.258539 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 03:34:08.259239 12708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:08.246419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:41723 (local address 127.27.40.131:60921)
0809 03:34:08.246793 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:08.246838 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:08.247480 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:08.247758 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:08.247762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:08.248021 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:34:08.248254 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:08.248258 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.248968 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.248969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:08.249876 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:08.249879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.250288 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.250289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:08.250424 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:08.252410 (+ 1986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:34:08.252443 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:34:08.254664 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:34:08.256881 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:08.256889 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:08.256891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:08.257249 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:08.257512 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:08.257513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:08.257514 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:08.257697 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:08.258039 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:34:08.258047 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:34:08.258251 (+ 204us) client_negotiation.cc:769] Sending connection context
0809 03:34:08.258752 (+ 501us) client_negotiation.cc:240] Negotiation successful
0809 03:34:08.258932 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":250,"threads_started":1}
I20260809 03:34:08.259845 12705 heartbeater.cc:347] Connected to a master server at 127.27.40.190:41723
I20260809 03:34:08.259971 12705 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:08.260174 12705 heartbeater.cc:511] Master 127.27.40.190:41723 requested a full tablet report, sending...
I20260809 03:34:08.260588 11993 ts_manager.cc:194] Registered new tserver with Master: 2b166b2070fb428bbde43505c54cc594 (127.27.40.131:45765)
I20260809 03:34:08.260964 11993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:60921
I20260809 03:34:08.263273 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:34:08.274025 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:08.265219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46848 (local address 127.27.40.190:41723)
0809 03:34:08.265296 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:08.265298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:08.265465 (+ 167us) server_negotiation.cc:407] Connection header received
0809 03:34:08.266493 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:08.266494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:08.266507 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:08.266579 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:08.267229 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.267712 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.268490 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.268702 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.268792 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:08.270601 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:08.270603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:08.270604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:08.270611 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:08.272635 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:08.273067 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:08.273068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:08.273069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:08.273134 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:08.273360 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:08.273361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:08.273362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:08.273485 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:08.273604 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:08.273850 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 03:34:08.273929 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:34:08.276240 11993 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46848:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:34:08.278553 11993 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:34:08.292083 12725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:08.285897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:37637 (local address 127.0.0.1:43298)
0809 03:34:08.287251 (+ 1354us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:08.287283 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:08.287383 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:08.287740 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:08.287743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:08.287745 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:34:08.287830 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:08.287832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.289260 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.289261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:08.290529 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:08.290532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.291766 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.291767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:08.291852 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:08.291863 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 03:34:08.291950 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 03:34:08.292001 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1318,"thread_start_us":175,"threads_started":1}
I20260809 03:34:08.292378 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:08.286533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43298 (local address 127.27.40.129:37637)
0809 03:34:08.287043 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:08.287048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:08.287366 (+ 318us) server_negotiation.cc:407] Connection header received
0809 03:34:08.287519 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:08.287523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:08.287552 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:08.287652 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:34:08.287929 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.289173 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.290631 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.291607 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.291673 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:08.291732 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:08.292208 (+ 476us) server_negotiation.cc:299] Negotiation successful
0809 03:34:08.292258 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":54,"threads_started":1}
I20260809 03:34:08.293092 12214 tablet_service.cc:1467] Processing CreateTablet for tablet c286ce01814d46a790ce162a9a94b577 (DEFAULT_TABLE table=test-table [id=7cc4de7df8a44806b911745f72c6dbcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:34:08.293491 12727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:08.285903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:43345 (local address 127.0.0.1:53160)
0809 03:34:08.287595 (+ 1692us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:08.287605 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:08.287650 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:08.288178 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:08.288179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:08.288180 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:34:08.288215 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:08.288216 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.289846 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.289847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:08.291213 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:08.291215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.293215 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.293217 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:08.293287 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:08.293292 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:34:08.293359 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 03:34:08.293400 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1654,"thread_start_us":194,"threads_started":1}
I20260809 03:34:08.293524 12214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c286ce01814d46a790ce162a9a94b577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:34:08.293900 12728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:08.286284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53160 (local address 127.27.40.130:43345)
0809 03:34:08.287171 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:08.287175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:08.287705 (+ 530us) server_negotiation.cc:407] Connection header received
0809 03:34:08.287775 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:08.287777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:08.287793 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:08.287868 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:34:08.288286 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.289758 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.291396 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.293085 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.293174 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:08.293364 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:08.293411 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 03:34:08.293461 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":849,"thread_start_us":141,"threads_started":1}
I20260809 03:34:08.294958 12418 tablet_service.cc:1467] Processing CreateTablet for tablet c286ce01814d46a790ce162a9a94b577 (DEFAULT_TABLE table=test-table [id=7cc4de7df8a44806b911745f72c6dbcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:34:08.295378 12418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c286ce01814d46a790ce162a9a94b577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:34:08.299753 12735 tablet_bootstrap.cc:492] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb: Bootstrap starting.
I20260809 03:34:08.299889 12732 tablet_bootstrap.cc:492] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd: Bootstrap starting.
I20260809 03:34:08.300786 12729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:08.286261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:45765 (local address 127.0.0.1:57402)
0809 03:34:08.288048 (+ 1787us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:08.288067 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:08.288122 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:08.294869 (+ 6747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:08.294870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:08.294879 (+ 9us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:34:08.294922 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:08.294924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.297011 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.297022 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:08.298915 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:08.298918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.300541 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.300543 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:08.300609 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:08.300614 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:34:08.300663 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 03:34:08.300700 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1745,"thread_start_us":201,"threads_started":1}
I20260809 03:34:08.301617 12732 tablet_bootstrap.cc:654] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd: Neither blocks nor log segments found. Creating new log.
I20260809 03:34:08.301702 12735 tablet_bootstrap.cc:654] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb: Neither blocks nor log segments found. Creating new log.
I20260809 03:34:08.301654 12731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:08.286804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57402 (local address 127.27.40.131:45765)
0809 03:34:08.294609 (+ 7805us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:08.294613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:08.294636 (+ 23us) server_negotiation.cc:407] Connection header received
0809 03:34:08.294698 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:08.294700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:08.294716 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:08.295021 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:34:08.295255 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.296912 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.299139 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.300431 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.301274 (+ 843us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:08.301427 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:08.301455 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 03:34:08.301493 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7731,"thread_start_us":142,"threads_started":1}
I20260809 03:34:08.302505 12732 log.cc:824] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd: Log is configured to *not* fsync() on all Append() calls
I20260809 03:34:08.302531 12735 log.cc:824] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb: Log is configured to *not* fsync() on all Append() calls
I20260809 03:34:08.302803 12594 tablet_service.cc:1467] Processing CreateTablet for tablet c286ce01814d46a790ce162a9a94b577 (DEFAULT_TABLE table=test-table [id=7cc4de7df8a44806b911745f72c6dbcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:34:08.303184 12594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c286ce01814d46a790ce162a9a94b577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:34:08.303619 12732 tablet_bootstrap.cc:492] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd: No bootstrap required, opened a new log
I20260809 03:34:08.303684 12732 ts_tablet_manager.cc:1397] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 03:34:08.303879 12735 tablet_bootstrap.cc:492] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb: No bootstrap required, opened a new log
I20260809 03:34:08.303996 12735 ts_tablet_manager.cc:1397] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 03:34:08.308085 12735 raft_consensus.cc:348] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363af8add0746eab2d905afed91c8fd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37637 } } peers { permanent_uuid: "6ebcd3a95a234d94aa6a621782b80bcb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 43345 } } peers { permanent_uuid: "2b166b2070fb428bbde43505c54cc594" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45765 } }
I20260809 03:34:08.308347 12735 raft_consensus.cc:374] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:34:08.308429 12735 raft_consensus.cc:729] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ebcd3a95a234d94aa6a621782b80bcb, State: Initialized, Role: FOLLOWER
I20260809 03:34:08.308666 12735 consensus_queue.cc:260] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363af8add0746eab2d905afed91c8fd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37637 } } peers { permanent_uuid: "6ebcd3a95a234d94aa6a621782b80bcb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 43345 } } peers { permanent_uuid: "2b166b2070fb428bbde43505c54cc594" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45765 } }
I20260809 03:34:08.308765 12732 raft_consensus.cc:348] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363af8add0746eab2d905afed91c8fd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37637 } } peers { permanent_uuid: "6ebcd3a95a234d94aa6a621782b80bcb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 43345 } } peers { permanent_uuid: "2b166b2070fb428bbde43505c54cc594" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45765 } }
I20260809 03:34:08.308982 12732 raft_consensus.cc:374] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:34:08.309024 12732 raft_consensus.cc:729] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3363af8add0746eab2d905afed91c8fd, State: Initialized, Role: FOLLOWER
I20260809 03:34:08.309088 12735 ts_tablet_manager.cc:1428] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260809 03:34:08.309134 12732 consensus_queue.cc:260] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363af8add0746eab2d905afed91c8fd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37637 } } peers { permanent_uuid: "6ebcd3a95a234d94aa6a621782b80bcb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 43345 } } peers { permanent_uuid: "2b166b2070fb428bbde43505c54cc594" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45765 } }
I20260809 03:34:08.309213 12510 heartbeater.cc:503] Master 127.27.40.190:41723 was elected leader, sending a full tablet report...
I20260809 03:34:08.309459 12732 ts_tablet_manager.cc:1428] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260809 03:34:08.309633 12306 heartbeater.cc:503] Master 127.27.40.190:41723 was elected leader, sending a full tablet report...
I20260809 03:34:08.311594 12748 tablet_bootstrap.cc:492] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594: Bootstrap starting.
I20260809 03:34:08.312849 12748 tablet_bootstrap.cc:654] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594: Neither blocks nor log segments found. Creating new log.
I20260809 03:34:08.313123 12748 log.cc:824] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594: Log is configured to *not* fsync() on all Append() calls
I20260809 03:34:08.314052 12748 tablet_bootstrap.cc:492] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594: No bootstrap required, opened a new log
I20260809 03:34:08.314138 12748 ts_tablet_manager.cc:1397] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:34:08.316435 12748 raft_consensus.cc:348] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363af8add0746eab2d905afed91c8fd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37637 } } peers { permanent_uuid: "6ebcd3a95a234d94aa6a621782b80bcb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 43345 } } peers { permanent_uuid: "2b166b2070fb428bbde43505c54cc594" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45765 } }
I20260809 03:34:08.316632 12748 raft_consensus.cc:374] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:34:08.316690 12748 raft_consensus.cc:729] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b166b2070fb428bbde43505c54cc594, State: Initialized, Role: FOLLOWER
I20260809 03:34:08.316839 12748 consensus_queue.cc:260] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363af8add0746eab2d905afed91c8fd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37637 } } peers { permanent_uuid: "6ebcd3a95a234d94aa6a621782b80bcb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 43345 } } peers { permanent_uuid: "2b166b2070fb428bbde43505c54cc594" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45765 } }
I20260809 03:34:08.317157 12748 ts_tablet_manager.cc:1428] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:34:08.317222 12705 heartbeater.cc:503] Master 127.27.40.190:41723 was elected leader, sending a full tablet report...
W20260809 03:34:08.421417 12310 tablet.cc:2369] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:34:08.453577 12511 tablet.cc:2369] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 03:34:08 dist-test-slave-gkw7 krb5kdc[11293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246447, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
W20260809 03:34:08.496295 12706 tablet.cc:2369] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:34:08.630101 12741 raft_consensus.cc:482] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:34:08.630300 12741 raft_consensus.cc:504] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363af8add0746eab2d905afed91c8fd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37637 } } peers { permanent_uuid: "6ebcd3a95a234d94aa6a621782b80bcb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 43345 } } peers { permanent_uuid: "2b166b2070fb428bbde43505c54cc594" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45765 } }
I20260809 03:34:08.630832 12741 leader_election.cc:288] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ebcd3a95a234d94aa6a621782b80bcb (127.27.40.130:43345), 2b166b2070fb428bbde43505c54cc594 (127.27.40.131:45765)
I20260809 03:34:08.636339 12728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:08.631203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:37187 (local address 127.27.40.130:43345)
0809 03:34:08.631275 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:08.631277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:08.631694 (+ 417us) server_negotiation.cc:407] Connection header received
0809 03:34:08.631871 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:08.631872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:08.631884 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:08.631918 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:34:08.632408 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.633173 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.635235 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.635870 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.635910 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:08.635916 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:08.636222 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 03:34:08.636264 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:34:08.636339 12731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:08.631188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:55625 (local address 127.27.40.131:45765)
0809 03:34:08.631261 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:08.631264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:08.631744 (+ 480us) server_negotiation.cc:407] Connection header received
0809 03:34:08.631871 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:08.631872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:08.631882 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:08.631916 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:34:08.632383 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.633181 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.635235 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.635832 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.635884 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:08.635897 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:08.636229 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 03:34:08.636271 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 03:34:08.636677 12756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:08.630996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:43345 (local address 127.27.40.129:37187)
0809 03:34:08.631567 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:08.631606 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:08.631783 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:08.631993 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:08.631994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:08.631996 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:34:08.632320 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:08.632321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.633323 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.633324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:08.635124 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:08.635128 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.636003 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.636004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:08.636124 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:08.636129 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:34:08.636469 (+ 340us) client_negotiation.cc:240] Negotiation successful
0809 03:34:08.636526 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":109,"thread_start_us":434,"threads_started":1}
I20260809 03:34:08.637002 12757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:08.631020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:45765 (local address 127.27.40.129:55625)
0809 03:34:08.631674 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:08.631683 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:08.631783 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:08.631993 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:08.631994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:08.631997 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:34:08.632265 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:08.632269 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.633382 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.633383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:08.635124 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:08.635128 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.636006 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.636007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:08.636144 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:08.636147 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:34:08.636866 (+ 719us) client_negotiation.cc:240] Negotiation successful
0809 03:34:08.636911 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":82,"thread_start_us":410,"threads_started":1}
I20260809 03:34:08.637034 12446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c286ce01814d46a790ce162a9a94b577" candidate_uuid: "3363af8add0746eab2d905afed91c8fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ebcd3a95a234d94aa6a621782b80bcb" is_pre_election: true
I20260809 03:34:08.637202 12446 raft_consensus.cc:2454] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3363af8add0746eab2d905afed91c8fd in term 0.
I20260809 03:34:08.637565 12140 leader_election.cc:302] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3363af8add0746eab2d905afed91c8fd, 6ebcd3a95a234d94aa6a621782b80bcb; no voters:
I20260809 03:34:08.637729 12741 raft_consensus.cc:2781] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:34:08.637784 12741 raft_consensus.cc:482] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:34:08.637714 12640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c286ce01814d46a790ce162a9a94b577" candidate_uuid: "3363af8add0746eab2d905afed91c8fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b166b2070fb428bbde43505c54cc594" is_pre_election: true
I20260809 03:34:08.637813 12741 raft_consensus.cc:3037] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:34:08.637877 12640 raft_consensus.cc:2454] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3363af8add0746eab2d905afed91c8fd in term 0.
I20260809 03:34:08.638837 12741 raft_consensus.cc:504] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363af8add0746eab2d905afed91c8fd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37637 } } peers { permanent_uuid: "6ebcd3a95a234d94aa6a621782b80bcb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 43345 } } peers { permanent_uuid: "2b166b2070fb428bbde43505c54cc594" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45765 } }
I20260809 03:34:08.638989 12741 leader_election.cc:288] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [CANDIDATE]: Term 1 election: Requested vote from peers 6ebcd3a95a234d94aa6a621782b80bcb (127.27.40.130:43345), 2b166b2070fb428bbde43505c54cc594 (127.27.40.131:45765)
I20260809 03:34:08.639242 12640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c286ce01814d46a790ce162a9a94b577" candidate_uuid: "3363af8add0746eab2d905afed91c8fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b166b2070fb428bbde43505c54cc594"
I20260809 03:34:08.639340 12640 raft_consensus.cc:3037] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:34:08.639341 12446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c286ce01814d46a790ce162a9a94b577" candidate_uuid: "3363af8add0746eab2d905afed91c8fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ebcd3a95a234d94aa6a621782b80bcb"
I20260809 03:34:08.639421 12446 raft_consensus.cc:3037] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:34:08.640511 12446 raft_consensus.cc:2454] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3363af8add0746eab2d905afed91c8fd in term 1.
I20260809 03:34:08.640708 12140 leader_election.cc:302] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3363af8add0746eab2d905afed91c8fd, 6ebcd3a95a234d94aa6a621782b80bcb; no voters:
I20260809 03:34:08.640782 12640 raft_consensus.cc:2454] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3363af8add0746eab2d905afed91c8fd in term 1.
I20260809 03:34:08.640820 12741 raft_consensus.cc:2781] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:34:08.641055 12741 raft_consensus.cc:686] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [term 1 LEADER]: Becoming Leader. State: Replica: 3363af8add0746eab2d905afed91c8fd, State: Running, Role: LEADER
I20260809 03:34:08.641189 12741 consensus_queue.cc:237] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363af8add0746eab2d905afed91c8fd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37637 } } peers { permanent_uuid: "6ebcd3a95a234d94aa6a621782b80bcb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 43345 } } peers { permanent_uuid: "2b166b2070fb428bbde43505c54cc594" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45765 } }
I20260809 03:34:08.642223 11993 catalog_manager.cc:5495] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 3363af8add0746eab2d905afed91c8fd (127.27.40.129). New cstate: current_term: 1 leader_uuid: "3363af8add0746eab2d905afed91c8fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363af8add0746eab2d905afed91c8fd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ebcd3a95a234d94aa6a621782b80bcb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 43345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b166b2070fb428bbde43505c54cc594" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260809 03:34:08.732785 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:08.730220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43314 (local address 127.27.40.129:37637)
0809 03:34:08.730264 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:08.730266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:08.730276 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:34:08.730364 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:08.730365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:08.730389 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:08.730476 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:34:08.730610 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.731185 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.731963 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:08.732180 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:08.732255 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:08.732433 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:34:08.732547 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:34:08.732649 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:08.732669 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 03:34:08.732691 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 03:34:08.735006 12640 raft_consensus.cc:1261] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594 [term 1 FOLLOWER]: Refusing update from remote peer 3363af8add0746eab2d905afed91c8fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:34:08.735083 12446 raft_consensus.cc:1261] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb [term 1 FOLLOWER]: Refusing update from remote peer 3363af8add0746eab2d905afed91c8fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:34:08.735657 12758 consensus_queue.cc:1035] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ebcd3a95a234d94aa6a621782b80bcb" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 43345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:34:08.735843 12741 consensus_queue.cc:1035] T c286ce01814d46a790ce162a9a94b577 P 3363af8add0746eab2d905afed91c8fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b166b2070fb428bbde43505c54cc594" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:34:08.739876 12765 mvcc.cc:205] Tried to move back new op lower bound from 7316465454015520768 to 7316465453634998272. Current Snapshot: MvccSnapshot[applied={T|T < 7316465454015520768}]
I20260809 03:34:08.740316 12762 mvcc.cc:205] Tried to move back new op lower bound from 7316465454015520768 to 7316465453634998272. Current Snapshot: MvccSnapshot[applied={T|T < 7316465454015520768}]
I20260809 03:34:08.740448 12764 mvcc.cc:205] Tried to move back new op lower bound from 7316465454015520768 to 7316465453634998272. Current Snapshot: MvccSnapshot[applied={T|T < 7316465454015520768 or (T in {7316465454015520768})}]
I20260809 03:34:08.743896 11990 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:34:08.743986 11990 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:34:08.744810 11990 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da7e3f2652274b74bd567ac2a623e7e0: Sending DeleteTablet for 3 replicas of tablet c286ce01814d46a790ce162a9a94b577
I20260809 03:34:08.745177 12214 tablet_service.cc:1514] Processing DeleteTablet for tablet c286ce01814d46a790ce162a9a94b577 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:34:08 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:43298
I20260809 03:34:08.745177 12418 tablet_service.cc:1514] Processing DeleteTablet for tablet c286ce01814d46a790ce162a9a94b577 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:34:08 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:53160
I20260809 03:34:08.745316 12594 tablet_service.cc:1514] Processing DeleteTablet for tablet c286ce01814d46a790ce162a9a94b577 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:34:08 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:57402
I20260809 03:34:08.746119 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 12113
I20260809 03:34:08.753728 12776 tablet_replica.cc:330] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb: stopping tablet replica
I20260809 03:34:08.753885 12776 raft_consensus.cc:2229] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:34:08.754037 12776 raft_consensus.cc:2258] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:34:08.754518 12776 ts_tablet_manager.cc:1905] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:34:08.756197 12776 ts_tablet_manager.cc:1918] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:34:08.756294 12776 log.cc:1196] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/c286ce01814d46a790ce162a9a94b577
I20260809 03:34:08.756542 12776 ts_tablet_manager.cc:1939] T c286ce01814d46a790ce162a9a94b577 P 6ebcd3a95a234d94aa6a621782b80bcb: Deleting consensus metadata
I20260809 03:34:08.757021 11967 catalog_manager.cc:4841] TS 6ebcd3a95a234d94aa6a621782b80bcb (127.27.40.130:43345): tablet c286ce01814d46a790ce162a9a94b577 (table test-table [id=7cc4de7df8a44806b911745f72c6dbcf]) successfully deleted
W20260809 03:34:08.758354 11972 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.40.129:37637 (error 108)
I20260809 03:34:08.765637 12777 tablet_replica.cc:330] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594: stopping tablet replica
I20260809 03:34:08.765784 12777 raft_consensus.cc:2229] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:34:08.765913 12777 raft_consensus.cc:2258] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 03:34:08.766019 12729 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:34:08.758772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:37637 (local address 127.0.0.1:43330)
0809 03:34:08.758868 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:08.758883 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:08.758940 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:08.765946 (+ 7006us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:37637: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":70}
W20260809 03:34:08.766232 11972 catalog_manager.cc:4584] TS 3363af8add0746eab2d905afed91c8fd (127.27.40.129:37637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c286ce01814d46a790ce162a9a94b577: Network error: Client connection negotiation failed: client connection to 127.27.40.129:37637: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 03:34:08.766435 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 12318
I20260809 03:34:08.779397 12777 ts_tablet_manager.cc:1905] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:34:08.781829 12777 ts_tablet_manager.cc:1918] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:34:08.781929 12777 log.cc:1196] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/c286ce01814d46a790ce162a9a94b577
I20260809 03:34:08.782225 12777 ts_tablet_manager.cc:1939] T c286ce01814d46a790ce162a9a94b577 P 2b166b2070fb428bbde43505c54cc594: Deleting consensus metadata
I20260809 03:34:08.782876 11973 catalog_manager.cc:4841] TS 2b166b2070fb428bbde43505c54cc594 (127.27.40.131:45765): tablet c286ce01814d46a790ce162a9a94b577 (table test-table [id=7cc4de7df8a44806b911745f72c6dbcf]) successfully deleted
I20260809 03:34:08.784493 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 12519
I20260809 03:34:08.794780 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 11940
I20260809 03:34:08.805384 27810 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 03:34:09.149462 27810 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 03:34:09.149688 27810 mini_ranger.cc:66] Stopping Ranger...
I20260809 03:34:09.505102 27810 mini_ranger.cc:68] Stopped Ranger
2026-08-09 03:34:09.505 UTC [11337] LOG: received smart shutdown request
2026-08-09 03:34:09.509 UTC [11337] LOG: background worker "logical replication launcher" (PID 11354) exited with exit code 1
2026-08-09 03:34:09.509 UTC [11348] LOG: shutting down
2026-08-09 03:34:09.677 UTC [11337] LOG: database system is shut down
2026-08-09T03:34:09Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48535 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:34:09 dist-test-slave-gkw7 krb5kdc[12787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:34:09 dist-test-slave-gkw7 krb5kdc[12787](info): set up 2 sockets
Aug 09 03:34:09 dist-test-slave-gkw7 krb5kdc[12787](info): commencing operation
krb5kdc: starting...
W20260809 03:34:11.039352 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:34:11 dist-test-slave-gkw7 krb5kdc[12787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:34:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:34:11Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.27.40.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.27.40.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.27.40.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/rangeradmin_127.27.40.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.27.40.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/rangeradmin_127.27.40.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.27.40.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.27.40.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.27.40.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/rangerlookup_127.27.40.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.27.40.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/rangerlookup_127.27.40.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.40.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.27.40.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.40.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/HTTP_127.27.40.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.40.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/HTTP_127.27.40.148@KRBTEST.COM.keytab.
I20260809 03:34:11.215742 27810 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskcMSfqS/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/postgres -l logfile start
2026-08-09 03:34:13.671 UTC [12832] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 03:34:13.671 UTC [12832] LOG: listening on IPv4 address "127.27.40.148", port 36965
2026-08-09 03:34:13.674 UTC [12832] LOG: listening on Unix socket "/tmp/.s.PGSQL.36965"
2026-08-09 03:34:13.717 UTC [12835] LOG: database system was shut down at 2026-08-09 03:34:12 UTC
2026-08-09 03:34:13.721 UTC [12832] LOG: database system is ready to accept connections
I20260809 03:34:14.864886 27810 mini_postgres.cc:96] Postgres bound to 36965
2026-08-09 03:34:14.871 UTC [12844] FATAL: database "slave" does not exist
127.27.40.148:36965 - accepting connections
I20260809 03:34:14.872416 27810 mini_ranger.cc:161] Starting Ranger...
I20260809 03:34:14.887697 27810 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 03:34:15.374229 27810 mini_ranger.cc:87] Created ranger Postgres database
I20260809 03:34:15.374327 27810 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-admin
2026-08-09 03:34:15,660 [I] DB FLAVOR :POSTGRES
2026-08-09 03:34:15,661 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 03:34:15,661 [I] Checking connection..
2026-08-09 03:34:15,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 03:34:15,915 [I] Checking connection passed.
2026-08-09 03:34:15,915 [I] --------- Verifying version history table ---------
2026-08-09 03:34:15,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:34:16,203 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 03:34:16,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:34:16,465 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 03:34:16,465 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 03:34:16,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 03:34:16.710 UTC [12937] WARNING: there is no transaction in progress
2026-08-09 03:34:16,719 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 03:34:16,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:34:16,973 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 03:34:16,973 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 03:34:16,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 03:34:17,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 03:34:17,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gkw7.c.gcp-upstream.internal','N') ;"
2026-08-09 03:34:17,753 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 03:34:18,019 [I] Table x_portal_user does not exist in database ranger
2026-08-09 03:34:18,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 03:34:18,277 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 03:34:18,277 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 03:34:18,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 03:34:18,769 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 03:34:18,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 03:34:19.780 UTC [13089] WARNING: there is no transaction in progress
2026-08-09 03:34:19.797 UTC [13089] WARNING: there is no transaction in progress
2026-08-09 03:34:19.813 UTC [13089] WARNING: there is no transaction in progress
2026-08-09 03:34:19.829 UTC [13089] WARNING: there is no transaction in progress
2026-08-09 03:34:19.849 UTC [13089] WARNING: there is no transaction in progress
2026-08-09 03:34:19.869 UTC [13089] WARNING: there is no transaction in progress
2026-08-09 03:34:19.926 UTC [13089] WARNING: there is no transaction in progress
2026-08-09 03:34:19.932 UTC [13089] WARNING: there is no transaction in progress
2026-08-09 03:34:19.946 UTC [13089] WARNING: there is no transaction in progress
2026-08-09 03:34:19.957 UTC [13089] WARNING: there is no transaction in progress
2026-08-09 03:34:19.967 UTC [13089] WARNING: there is no transaction in progress
2026-08-09 03:34:19.982 UTC [13089] WARNING: there is no transaction in progress
2026-08-09 03:34:19.991 UTC [13089] WARNING: there is no transaction in progress
2026-08-09 03:34:19.998 UTC [13089] WARNING: there is no transaction in progress
2026-08-09 03:34:20.003 UTC [13089] WARNING: there is no transaction in progress
2026-08-09 03:34:20.009 UTC [13089] WARNING: there is no transaction in progress
2026-08-09 03:34:20,645 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 03:34:20,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 03:34:20,917 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 03:34:20,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 03:34:21,181 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 03:34:21,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 03:34:21,433 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 03:34:21,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 03:34:21,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 03:34:21,942 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gkw7.c.gcp-upstream.internal';"
2026-08-09 03:34:22,187 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 03:34:22,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 03:34:22,437 [I] DB_PATCHES have already been applied
I20260809 03:34:22.443276 27810 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-admin:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskcMSfqS/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 03:34:22.443372 27810 mini_ranger.cc:193] Using host: 127.27.40.148
I20260809 03:34:22.447393 27810 mini_ranger.cc:227] Ranger admin URL: http://127.27.40.148:52043
Aug 09, 2026 3:34:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:34:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:34:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 3:34:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 3:34:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 3:34:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 3:34:23 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 03:34:23.771373 27810 mini_ranger.cc:160] Time spent starting Ranger: real 8.899s user 0.000s sys 0.011s
Aug 09, 2026 3:34:43 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 3:34:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 09, 2026 3:34:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 3:34:43 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 3:34:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 3:34:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 3:34:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 3:34:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 3:34:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 3:34:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 3:34:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 3:34:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 3:34:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 3:34:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 3:34:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 3:34:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 3:34:43 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 3:34:44 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 03:34:44.665269 27810 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.27.40.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.27.40.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.27.40.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/rangerkms_127.27.40.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.27.40.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/rangerkms_127.27.40.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.40.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.27.40.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.40.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/HTTP_127.27.40.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.40.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/HTTP_127.27.40.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 03:34:44 dist-test-slave-gkw7 krb5kdc[12787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246484, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 03:34:44.754721 27810 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 03:34:44.765144 27810 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 03:34:45.684590 27810 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 03:34:45.684682 27810 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-kms
I20260809 03:34:45.684705 27810 mini_ranger_kms.cc:227] Using postgres at 127.27.40.148:36965
2026-08-09 03:34:45,864 [I] DB FLAVOR :POSTGRES
2026-08-09 03:34:45,865 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 03:34:45,865 [I] Checking connection
2026-08-09 03:34:45,865 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 03:34:46,150 [I] connection success
2026-08-09 03:34:46,151 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 03:34:46,151 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 03:34:46,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 03:34:46,402 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 03:34:46,402 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 03:34:46,402 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 03:34:46,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.40.148:36965/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 03:34:46,664 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 03:34:47.148965 27810 mini_ranger_kms.cc:310] Created kms service
I20260809 03:34:47.366357 27810 mini_ranger_kms.cc:326] Created kudu user
I20260809 03:34:47.410868 27810 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 03:34:47 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246484, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
I20260809 03:34:47.839609 27810 mini_ranger_kms.cc:384] Added ranger policy
I20260809 03:34:47.839836 27810 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-kms:/tmp/dist-test-taskcMSfqS/build/release/bin/postgresql.jar:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskcMSfqS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 03:34:47.839862 27810 mini_ranger_kms.cc:240] Using host: 127.27.40.148
I20260809 03:34:47.845259 27810 mini_ranger_kms.cc:276] Ranger KMS PID: 13344
I20260809 03:34:47.845353 27810 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.27.40.148:54217
Aug 09, 2026 3:34:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:34:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:34:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 3:34:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 3:34:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 3:34:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 03:34:50 dist-test-slave-gkw7 krb5kdc[12787](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786246490, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.40.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:50.586645 27810 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.832s user 0.000s sys 0.163s
I20260809 03:34:50.586755 27810 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 03:34:50.586803 27810 mini_ranger_kms.cc:401] 127.27.40.148:54217/kms/v1/keys
Aug 09 03:34:50 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246490, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.40.148@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
Aug 09 03:34:51 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246490, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.40.148@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
Aug 09 03:34:53 dist-test-slave-gkw7 krb5kdc[12787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:34:53.582508 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:36523
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:36523
--ranger_config_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.40.148:54217/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:34:53.674556 13415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:53.674773 13415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:53.674794 13415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:53.675956 13415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:34:53.675998 13415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:53.676012 13415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:53.676024 13415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:34:53.676036 13415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:34:53.677659 13415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/wal
--ranger_kms_url=127.27.40.148:54217/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:36523
--ranger_config_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:36523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:34:53.677913 13415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:53.678164 13415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:53.683630 13421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:53.683626 13426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:53.683693 13422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:53.683730 13415 server_base.cc:1034] running on GCE node
I20260809 03:34:53.684166 13415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:53.684463 13415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:53.685609 13415 hybrid_clock.cc:648] HybridClock initialized: now 1786246493685600 us; error 54 us; skew 500 ppm
Aug 09 03:34:53 dist-test-slave-gkw7 krb5kdc[12787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246493, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:53.688118 13415 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:34:53.688616 13415 webserver.cc:460] Webserver started at http://127.27.40.190:38483/ using document root <none> and password file <none>
I20260809 03:34:53.688879 13415 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:53.688944 13415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:34:53.689172 13415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:34:53 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246493, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
I20260809 03:34:53.735571 13415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "787bb3590fef49d396d6b21187377426"
format_stamp: "Formatted at 2026-08-09 03:34:53 on dist-test-slave-gkw7"
server_key: "5e6b00b6a9edb59df50f910b5b526485"
server_key_iv: "fae27a0fbee0cc0c55fc4986c8668372"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:53.736085 13415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "787bb3590fef49d396d6b21187377426"
format_stamp: "Formatted at 2026-08-09 03:34:53 on dist-test-slave-gkw7"
server_key: "5e6b00b6a9edb59df50f910b5b526485"
server_key_iv: "fae27a0fbee0cc0c55fc4986c8668372"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:53.738200 13415 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 03:34:53.754830 13436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:53.755134 13415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 03:34:53.755261 13415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "787bb3590fef49d396d6b21187377426"
format_stamp: "Formatted at 2026-08-09 03:34:53 on dist-test-slave-gkw7"
server_key: "5e6b00b6a9edb59df50f910b5b526485"
server_key_iv: "fae27a0fbee0cc0c55fc4986c8668372"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:53.755365 13415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:53.769748 13415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:53.771942 13415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:53.772094 13415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:53.777169 13415 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:36523
I20260809 03:34:53.777259 13512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:36523 every 8 connection(s)
I20260809 03:34:53.777709 13415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:34:53.778465 13513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:34:53.779911 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 13415
I20260809 03:34:53.780030 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:34:53.780222 27810 external_mini_cluster.cc:1428] Setting key 74412a9c83c79fb7df25bb2171784eaf
I20260809 03:34:53.781415 13513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426: Bootstrap starting.
I20260809 03:34:53.782193 13513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426: Neither blocks nor log segments found. Creating new log.
I20260809 03:34:53.782549 13513 log.cc:824] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426: Log is configured to *not* fsync() on all Append() calls
I20260809 03:34:53.783402 13513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426: No bootstrap required, opened a new log
I20260809 03:34:53.785492 13513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787bb3590fef49d396d6b21187377426" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36523 } }
I20260809 03:34:53.785682 13513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:34:53.785718 13513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787bb3590fef49d396d6b21187377426, State: Initialized, Role: FOLLOWER
I20260809 03:34:53.785897 13513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787bb3590fef49d396d6b21187377426" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36523 } }
I20260809 03:34:53.785977 13513 raft_consensus.cc:388] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:34:53.786006 13513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:34:53.786075 13513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:34:53.786794 13513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787bb3590fef49d396d6b21187377426" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36523 } }
I20260809 03:34:53.786896 13513 leader_election.cc:302] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 787bb3590fef49d396d6b21187377426; no voters:
I20260809 03:34:53.787092 13513 leader_election.cc:288] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:34:53.787153 13520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:34:53.787381 13520 raft_consensus.cc:686] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 1 LEADER]: Becoming Leader. State: Replica: 787bb3590fef49d396d6b21187377426, State: Running, Role: LEADER
I20260809 03:34:53.787402 13513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:34:53.787499 13520 consensus_queue.cc:237] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787bb3590fef49d396d6b21187377426" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36523 } }
I20260809 03:34:53.787900 13520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 787bb3590fef49d396d6b21187377426. Latest consensus state: current_term: 1 leader_uuid: "787bb3590fef49d396d6b21187377426" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787bb3590fef49d396d6b21187377426" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36523 } } }
I20260809 03:34:53.787986 13520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [sys.catalog]: This master's current role is: LEADER
I20260809 03:34:53.788079 13522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "787bb3590fef49d396d6b21187377426" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787bb3590fef49d396d6b21187377426" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36523 } } }
I20260809 03:34:53.788223 13522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [sys.catalog]: This master's current role is: LEADER
I20260809 03:34:53.788336 13536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 03:34:53 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:34:53.788872 13536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:34:53.789237 13513 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 03:34:53.797101 13536 catalog_manager.cc:1347] Generated new cluster ID: c44024de81f345d69b7ba656b06bebf3
I20260809 03:34:53.797250 13536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:34:53.797259 13518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:53.780797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43766 (local address 127.27.40.190:36523)
0809 03:34:53.781105 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:53.781116 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:53.781152 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:34:53.781720 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:53.781731 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:53.781820 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:53.781981 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:53.782866 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:53.784223 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:53.785426 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:53.786079 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:53.786196 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:53.789345 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:53.789349 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:53.789350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:53.789362 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:53.794388 (+ 5026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:53.794923 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:53.794927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:53.794927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:53.795128 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:53.795396 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:53.795397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:53.795398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:53.795786 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:53.796076 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:53.796180 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 03:34:53.796586 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":77,"threads_started":1}
I20260809 03:34:53.807716 13536 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:34:53.808274 13536 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:34:53.810514 13536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426: Generated new TSK 0
I20260809 03:34:53.810678 13536 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:34:54.584846 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.40.148:54217/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:36523
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:34:54.693940 13580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:54.694155 13580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:54.694180 13580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:54.695329 13580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:54.695374 13580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:54.695399 13580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:34:54.696879 13580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.27.40.148:54217/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:36523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:34:54.697134 13580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:54.697413 13580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:54.702875 13591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:54.702891 13592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:54.702982 13594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:54.703572 13580 server_base.cc:1034] running on GCE node
I20260809 03:34:54.703773 13580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:54.704027 13580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:54.705224 13580 hybrid_clock.cc:648] HybridClock initialized: now 1786246494705195 us; error 34 us; skew 500 ppm
Aug 09 03:34:54 dist-test-slave-gkw7 krb5kdc[12787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:54.708660 13580 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:34:54.709163 13580 webserver.cc:460] Webserver started at http://127.27.40.129:38961/ using document root <none> and password file <none>
I20260809 03:34:54.709465 13580 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:54.709554 13580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:34:54.709699 13580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:34:54 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
I20260809 03:34:54.728978 13580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "3e155abe61fd4d92ad0ce676f1905cdd"
format_stamp: "Formatted at 2026-08-09 03:34:54 on dist-test-slave-gkw7"
server_key: "8efad0f34035c540e375cf1816ef5e14"
server_key_iv: "62f47b59ecf8b0b4de2409dcebc7e8bc"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:54.729696 13580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "3e155abe61fd4d92ad0ce676f1905cdd"
format_stamp: "Formatted at 2026-08-09 03:34:54 on dist-test-slave-gkw7"
server_key: "8efad0f34035c540e375cf1816ef5e14"
server_key_iv: "62f47b59ecf8b0b4de2409dcebc7e8bc"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:54.736979 13580 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.000s
I20260809 03:34:54.749722 13607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:54.750015 13580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:34:54.750211 13580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "3e155abe61fd4d92ad0ce676f1905cdd"
format_stamp: "Formatted at 2026-08-09 03:34:54 on dist-test-slave-gkw7"
server_key: "8efad0f34035c540e375cf1816ef5e14"
server_key_iv: "62f47b59ecf8b0b4de2409dcebc7e8bc"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:54.750398 13580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:54.767462 13580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:54.769987 13580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:54.770174 13580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:54.770423 13580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:34:54.770831 13580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:34:54.770879 13580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:54.770910 13580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:34:54.770928 13580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:54.780618 13580 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:39741
I20260809 03:34:54.785219 13580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:34:54.787153 13786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:39741 every 8 connection(s)
I20260809 03:34:54.794993 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 13580
I20260809 03:34:54.795122 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:34:54.795317 27810 external_mini_cluster.cc:1428] Setting key a4d0fad96a1fef6ac95fe5323cc5743e
Aug 09 03:34:54 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:34:54.804509 13792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:54.787834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:36523 (local address 127.27.40.129:42947)
0809 03:34:54.789660 (+ 1826us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:54.789710 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:54.790339 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:54.790790 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:54.790796 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:54.791080 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:34:54.791400 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:54.791407 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:54.792439 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:54.792440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:54.793924 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:54.793928 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:54.794556 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:54.794558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:54.794721 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:54.798312 (+ 3591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:34:54.798324 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:34:54.800207 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:34:54.802594 (+ 2387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:54.802603 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:54.802606 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:54.802947 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:54.803259 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:54.803260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:54.803260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:54.803417 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:54.803749 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:34:54.803755 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:34:54.803941 (+ 186us) client_negotiation.cc:769] Sending connection context
0809 03:34:54.804053 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 03:34:54.804245 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1771,"thread_start_us":109,"threads_started":1}
I20260809 03:34:54.805613 13791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:54.787864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:42947 (local address 127.27.40.190:36523)
0809 03:34:54.790526 (+ 2662us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:54.790528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:54.790536 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:34:54.790563 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:54.790564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:54.790577 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:54.790673 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:54.791531 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:54.792311 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:54.794038 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:54.794401 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:54.794472 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:54.800330 (+ 5858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:54.800332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:54.800332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:54.800340 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:54.802484 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:54.803042 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:54.803044 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:54.803044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:54.803168 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:54.803503 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:54.803504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:54.803504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:54.803659 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:54.803722 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:54.805379 (+ 1657us) server_negotiation.cc:299] Negotiation successful
0809 03:34:54.805483 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2629,"thread_start_us":87,"threads_started":1}
I20260809 03:34:54.806258 13788 heartbeater.cc:347] Connected to a master server at 127.27.40.190:36523
I20260809 03:34:54.806382 13788 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:54.806584 13788 heartbeater.cc:511] Master 127.27.40.190:36523 requested a full tablet report, sending...
I20260809 03:34:54.807193 13464 ts_manager.cc:194] Registered new tserver with Master: 3e155abe61fd4d92ad0ce676f1905cdd (127.27.40.129:39741)
I20260809 03:34:54.808173 13464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:42947
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:34:54.841436 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.40.148:54217/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:36523
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:34:54 dist-test-slave-gkw7 krb5kdc[12787](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786246494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 03:34:54.968263 13801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:54.968494 13801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:54.968536 13801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:54.969820 13801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:54.969882 13801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:54.969928 13801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:34:54.971475 13801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.27.40.148:54217/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:36523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:34:54.971757 13801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:54.972039 13801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:54.977458 13810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:54.977666 13811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:54.977828 13815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:54.977900 13801 server_base.cc:1034] running on GCE node
I20260809 03:34:54.978250 13801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:54.978516 13801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:54.985908 13801 hybrid_clock.cc:648] HybridClock initialized: now 1786246494985867 us; error 51 us; skew 500 ppm
Aug 09 03:34:54 dist-test-slave-gkw7 krb5kdc[12787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:54.989355 13801 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:34:54.989993 13801 webserver.cc:460] Webserver started at http://127.27.40.130:33423/ using document root <none> and password file <none>
I20260809 03:34:54.990304 13801 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:54.990371 13801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:34:54.990510 13801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:34:54 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
I20260809 03:34:55.005497 13801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "039bca5c06cf4c5691e6a0b2afbbeb36"
format_stamp: "Formatted at 2026-08-09 03:34:54 on dist-test-slave-gkw7"
server_key: "5897e727244a1dd532fac679cd338120"
server_key_iv: "b2f01cb1880ee3c99ddfcb6d6d332807"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:55.006117 13801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "039bca5c06cf4c5691e6a0b2afbbeb36"
format_stamp: "Formatted at 2026-08-09 03:34:54 on dist-test-slave-gkw7"
server_key: "5897e727244a1dd532fac679cd338120"
server_key_iv: "b2f01cb1880ee3c99ddfcb6d6d332807"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:55.010134 13801 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.002s
I20260809 03:34:55.024593 13828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:55.024830 13801 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 03:34:55.024921 13801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "039bca5c06cf4c5691e6a0b2afbbeb36"
format_stamp: "Formatted at 2026-08-09 03:34:54 on dist-test-slave-gkw7"
server_key: "5897e727244a1dd532fac679cd338120"
server_key_iv: "b2f01cb1880ee3c99ddfcb6d6d332807"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:55.025009 13801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:55.051138 13801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:55.053617 13801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:55.053797 13801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:55.054095 13801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:34:55.054563 13801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:34:55.054620 13801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:55.054690 13801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:34:55.054723 13801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:55.062500 13801 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:44693
I20260809 03:34:55.062587 13988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:44693 every 8 connection(s)
I20260809 03:34:55.063055 13801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:34:55.066586 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 13801
I20260809 03:34:55.066731 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:34:55.066915 27810 external_mini_cluster.cc:1428] Setting key 72bdcd0d0e6037ff18d0ec53e719ab0a
Aug 09 03:34:55 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:34:55.077600 13791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:55.066302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:43353 (local address 127.27.40.190:36523)
0809 03:34:55.066376 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:55.066379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:55.066631 (+ 252us) server_negotiation.cc:407] Connection header received
0809 03:34:55.067434 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:55.067435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:55.067453 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:55.067486 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:55.068372 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:55.069132 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:55.070692 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:55.071040 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:55.071120 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:55.073916 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:55.073917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:55.073918 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:55.073927 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:55.075821 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:55.076350 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:55.076351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:55.076352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:55.076418 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:55.076749 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:55.076751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:55.076751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:55.076897 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:55.076942 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:55.077348 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 03:34:55.077447 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:34:55.078399 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:55.066289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:36523 (local address 127.27.40.130:43353)
0809 03:34:55.066467 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:55.066516 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:55.067296 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:55.067676 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:55.067680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:55.067943 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:34:55.068254 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:55.068260 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:55.069253 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:55.069254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:55.070581 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:55.070584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:55.071172 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:55.071173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:55.071339 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:55.071985 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:34:55.071999 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:34:55.073792 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:34:55.075919 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:55.075926 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:55.075930 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:55.076235 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:55.076493 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:55.076494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:55.076494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:55.076634 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:55.076995 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:34:55.077002 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:34:55.077176 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 03:34:55.077977 (+ 801us) client_negotiation.cc:240] Negotiation successful
0809 03:34:55.078154 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":93,"threads_started":1}
I20260809 03:34:55.079110 13989 heartbeater.cc:347] Connected to a master server at 127.27.40.190:36523
I20260809 03:34:55.079210 13989 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:55.079416 13989 heartbeater.cc:511] Master 127.27.40.190:36523 requested a full tablet report, sending...
I20260809 03:34:55.079867 13464 ts_manager.cc:194] Registered new tserver with Master: 039bca5c06cf4c5691e6a0b2afbbeb36 (127.27.40.130:44693)
I20260809 03:34:55.080372 13464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:43353
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:34:55.112517 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.40.148:54217/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:36523
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:34:55.226508 13996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:55.226802 13996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:55.226840 13996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:55.228523 13996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:55.228588 13996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:55.228631 13996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:34:55.230908 13996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.27.40.148:54217/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:36523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:34:55.231379 13996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:55.231737 13996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:55.238273 14004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:55.238494 14003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:55.238272 14006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:55.239439 13996 server_base.cc:1034] running on GCE node
I20260809 03:34:55.239675 13996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:55.239987 13996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:55.241191 13996 hybrid_clock.cc:648] HybridClock initialized: now 1786246495241167 us; error 38 us; skew 500 ppm
Aug 09 03:34:55 dist-test-slave-gkw7 krb5kdc[12787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246495, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:55.244632 13996 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:34:55.245209 13996 webserver.cc:460] Webserver started at http://127.27.40.131:42149/ using document root <none> and password file <none>
I20260809 03:34:55.245514 13996 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:55.245687 13996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:34:55.245908 13996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:34:55 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246495, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
I20260809 03:34:55.264052 13996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "ca7e1e468c72443698f8e7baf8cafbc6"
format_stamp: "Formatted at 2026-08-09 03:34:55 on dist-test-slave-gkw7"
server_key: "3c701ba368da7cdca17cdfcee4116016"
server_key_iv: "b910ccf81580e4ea5a921ac42dd83717"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:55.264549 13996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "ca7e1e468c72443698f8e7baf8cafbc6"
format_stamp: "Formatted at 2026-08-09 03:34:55 on dist-test-slave-gkw7"
server_key: "3c701ba368da7cdca17cdfcee4116016"
server_key_iv: "b910ccf81580e4ea5a921ac42dd83717"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:55.277803 13996 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.002s sys 0.000s
I20260809 03:34:55.294777 14015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:55.295153 13996 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:34:55.295270 13996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "ca7e1e468c72443698f8e7baf8cafbc6"
format_stamp: "Formatted at 2026-08-09 03:34:55 on dist-test-slave-gkw7"
server_key: "3c701ba368da7cdca17cdfcee4116016"
server_key_iv: "b910ccf81580e4ea5a921ac42dd83717"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:55.295372 13996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:55.310290 13996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:55.312614 13996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:55.312796 13996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:55.313051 13996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:34:55.313464 13996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:34:55.313632 13996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:55.313680 13996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:34:55.313705 13996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:55.319993 13996 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:44641
I20260809 03:34:55.320492 13996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:34:55.322474 14181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:44641 every 8 connection(s)
I20260809 03:34:55.330657 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 13996
I20260809 03:34:55.330791 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:34:55.331012 27810 external_mini_cluster.cc:1428] Setting key 165a318942f056f68b56f5e4ce3b4a3c
Aug 09 03:34:55 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246495, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:34:55.336963 13791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:55.323247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:44939 (local address 127.27.40.190:36523)
0809 03:34:55.323296 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:55.323298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:55.324030 (+ 732us) server_negotiation.cc:407] Connection header received
0809 03:34:55.324715 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:55.324717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:55.324730 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:55.324787 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:55.325765 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:55.326566 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:55.328174 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:55.328482 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:55.328557 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:55.333509 (+ 4952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:55.333511 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:55.333511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:55.333520 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:55.335306 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:55.335823 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:55.335824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:55.335824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:55.335887 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:55.336207 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:55.336207 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:55.336208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:55.336347 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:55.336426 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:55.336786 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 03:34:55.336872 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 03:34:55.337148 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:55.323160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:36523 (local address 127.27.40.131:44939)
0809 03:34:55.323850 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:55.323908 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:55.324574 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:55.325066 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:55.325070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:55.325342 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:34:55.325654 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:55.325661 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:55.326693 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:55.326694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:55.328046 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:55.328049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:55.328592 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:55.328593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:55.328740 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:55.330997 (+ 2257us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:34:55.331008 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:34:55.333394 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:34:55.335406 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:55.335415 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:55.335417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:55.335737 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:55.335974 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:55.335975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:55.335976 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:55.336127 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:55.336442 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:34:55.336449 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:34:55.336624 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 03:34:55.336748 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 03:34:55.336912 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":89,"threads_started":1}
I20260809 03:34:55.337947 14182 heartbeater.cc:347] Connected to a master server at 127.27.40.190:36523
I20260809 03:34:55.338060 14182 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:55.338275 14182 heartbeater.cc:511] Master 127.27.40.190:36523 requested a full tablet report, sending...
I20260809 03:34:55.338727 13464 ts_manager.cc:194] Registered new tserver with Master: ca7e1e468c72443698f8e7baf8cafbc6 (127.27.40.131:44641)
I20260809 03:34:55.339205 13464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:44939
I20260809 03:34:55.342689 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:34:55.343362 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 13580
I20260809 03:34:55.378175 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 13801
I20260809 03:34:55.403308 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 13996
Aug 09 03:34:55 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
I20260809 03:34:55.426065 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 13415
I20260809 03:34:55.447328 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:36523
--webserver_interface=127.27.40.190
--webserver_port=38483
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:36523
--ranger_config_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.40.148:54217/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.40.148:54217/kms with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:34:55.538440 14185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:55.538637 14185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:55.538658 14185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:55.539808 14185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:34:55.539848 14185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:55.539875 14185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:55.539888 14185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:34:55.539943 14185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:34:55.541517 14185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.40.148:54217/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:36523
--ranger_config_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:36523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=38483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:34:55.541834 14185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:55.542058 14185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:34:55.547333 14185 server_base.cc:1034] running on GCE node
W20260809 03:34:55.547382 14195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:55.547382 14194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:55.547482 14197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:55.547770 14185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:55.548063 14185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:55.549254 14185 hybrid_clock.cc:648] HybridClock initialized: now 1786246495549233 us; error 41 us; skew 500 ppm
Aug 09 03:34:55 dist-test-slave-gkw7 krb5kdc[12787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246495, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:55.551898 14185 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:34:55.552281 14185 webserver.cc:460] Webserver started at http://127.27.40.190:38483/ using document root <none> and password file <none>
I20260809 03:34:55.552465 14185 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:55.552515 14185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 03:34:55 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246495, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
I20260809 03:34:55.598544 14185 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:34:55.599436 14210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:55.599665 14185 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:34:55.599792 14185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "787bb3590fef49d396d6b21187377426"
format_stamp: "Formatted at 2026-08-09 03:34:53 on dist-test-slave-gkw7"
server_key: "5e6b00b6a9edb59df50f910b5b526485"
server_key_iv: "fae27a0fbee0cc0c55fc4986c8668372"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:55.600167 14185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:55.612301 14185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:55.614670 14185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:55.614841 14185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:55.619495 14185 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:36523
I20260809 03:34:55.619527 14278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:36523 every 8 connection(s)
I20260809 03:34:55.619954 14185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:34:55.621234 14281 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:34:55.621899 14281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426: Bootstrap starting.
I20260809 03:34:55.624634 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 14185
I20260809 03:34:55.624938 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.40.148:54217/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:39741
--local_ip_for_outbound_sockets=127.27.40.129
--tserver_master_addrs=127.27.40.190:36523
--webserver_port=38961
--webserver_interface=127.27.40.129
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.40.148:54217/kms with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:34:55.630373 14281 log.cc:824] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426: Log is configured to *not* fsync() on all Append() calls
I20260809 03:34:55.632313 14281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:34:55.632637 14281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426: Bootstrap complete.
I20260809 03:34:55.635175 14281 raft_consensus.cc:348] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787bb3590fef49d396d6b21187377426" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36523 } }
I20260809 03:34:55.635406 14281 raft_consensus.cc:729] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787bb3590fef49d396d6b21187377426, State: Initialized, Role: FOLLOWER
I20260809 03:34:55.635591 14281 consensus_queue.cc:260] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787bb3590fef49d396d6b21187377426" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36523 } }
I20260809 03:34:55.635666 14281 raft_consensus.cc:388] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:34:55.635720 14281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:34:55.635771 14281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:34:55.636758 14281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787bb3590fef49d396d6b21187377426" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36523 } }
I20260809 03:34:55.636847 14281 leader_election.cc:302] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 787bb3590fef49d396d6b21187377426; no voters:
I20260809 03:34:55.637101 14281 leader_election.cc:288] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 03:34:55.637136 14286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 03:34:55.637243 14286 raft_consensus.cc:686] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [term 2 LEADER]: Becoming Leader. State: Replica: 787bb3590fef49d396d6b21187377426, State: Running, Role: LEADER
I20260809 03:34:55.637383 14281 sys_catalog.cc:564] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:34:55.637416 14286 consensus_queue.cc:237] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787bb3590fef49d396d6b21187377426" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36523 } }
I20260809 03:34:55.637854 14287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "787bb3590fef49d396d6b21187377426" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787bb3590fef49d396d6b21187377426" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36523 } } }
I20260809 03:34:55.637939 14287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [sys.catalog]: This master's current role is: LEADER
I20260809 03:34:55.638254 14288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 787bb3590fef49d396d6b21187377426. Latest consensus state: current_term: 2 leader_uuid: "787bb3590fef49d396d6b21187377426" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787bb3590fef49d396d6b21187377426" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 36523 } } }
I20260809 03:34:55.638319 14288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426 [sys.catalog]: This master's current role is: LEADER
I20260809 03:34:55.638547 14297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:34:55.639039 14297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:34:55.639321 14297 catalog_manager.cc:1259] Loaded cluster ID: c44024de81f345d69b7ba656b06bebf3
I20260809 03:34:55.639400 14297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:34:55.639421 14281 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 03:34:55.640607 14297 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:34:55.640784 14297 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426: Loaded TSK: 0
I20260809 03:34:55.640998 14297 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 03:34:55.736058 14284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:55.736277 14284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:55.736317 14284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:55.737841 14284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:55.737929 14284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:55.737985 14284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:34:55.739656 14284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.40.148:54217/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:39741
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=38961
--tserver_master_addrs=127.27.40.190:36523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:34:55.739923 14284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:55.740203 14284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:55.745445 14336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:55.745445 14337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:55.745460 14339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:55.746217 14284 server_base.cc:1034] running on GCE node
I20260809 03:34:55.746433 14284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:55.746675 14284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:55.754063 14284 hybrid_clock.cc:648] HybridClock initialized: now 1786246495752661 us; error 1422 us; skew 500 ppm
Aug 09 03:34:55 dist-test-slave-gkw7 krb5kdc[12787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246495, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:55.756942 14284 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:34:55.757517 14284 webserver.cc:460] Webserver started at http://127.27.40.129:38961/ using document root <none> and password file <none>
I20260809 03:34:55.757889 14284 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:55.758008 14284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 03:34:55 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246495, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
I20260809 03:34:55.787670 14284 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.000s
I20260809 03:34:55.788966 14350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:55.789150 14284 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 03:34:55.789255 14284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "3e155abe61fd4d92ad0ce676f1905cdd"
format_stamp: "Formatted at 2026-08-09 03:34:54 on dist-test-slave-gkw7"
server_key: "8efad0f34035c540e375cf1816ef5e14"
server_key_iv: "62f47b59ecf8b0b4de2409dcebc7e8bc"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:55.789780 14284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:55.804740 14284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:55.807312 14284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:55.807488 14284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:55.807767 14284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:34:55.808166 14284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:34:55.808231 14284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:55.808266 14284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:34:55.808286 14284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:55.815537 14284 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:39741
I20260809 03:34:55.815688 14509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:39741 every 8 connection(s)
I20260809 03:34:55.816066 14284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
Aug 09 03:34:55 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246495, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:34:55.824553 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 14284
I20260809 03:34:55.824873 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.40.148:54217/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:44693
--local_ip_for_outbound_sockets=127.27.40.130
--tserver_master_addrs=127.27.40.190:36523
--webserver_port=33423
--webserver_interface=127.27.40.130
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.40.148:54217/kms with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:34:55.829442 14513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:55.817064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:36523 (local address 127.27.40.129:47087)
0809 03:34:55.817440 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:55.817488 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:55.818155 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:55.818628 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:55.818633 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:55.818947 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:34:55.819233 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:55.819241 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:55.820604 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:55.820606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:55.821933 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:55.821935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:55.822560 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:55.822562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:55.822713 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:55.823273 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:34:55.823283 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:34:55.824840 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:34:55.827496 (+ 2656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:55.827503 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:55.827505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:55.827812 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:55.828097 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:55.828098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:55.828098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:55.828242 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:55.828750 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:34:55.828756 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:34:55.828920 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 03:34:55.829043 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 03:34:55.829211 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":87,"threads_started":1}
I20260809 03:34:55.829563 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:55.817185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:47087 (local address 127.27.40.190:36523)
0809 03:34:55.817427 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:55.817435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:55.817635 (+ 200us) server_negotiation.cc:407] Connection header received
0809 03:34:55.818375 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:55.818381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:55.818427 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:55.818540 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:55.819371 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:55.820474 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:55.822016 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:55.822438 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:55.822522 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:55.825186 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:55.825190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:55.825190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:55.825201 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:55.827392 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:55.827911 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:55.827915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:55.827915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:55.828014 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:55.828321 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:55.828322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:55.828322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:55.828655 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:55.828745 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:55.829127 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 03:34:55.829270 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":101,"threads_started":1}
I20260809 03:34:55.830610 14511 heartbeater.cc:347] Connected to a master server at 127.27.40.190:36523
I20260809 03:34:55.830725 14511 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:55.830926 14511 heartbeater.cc:511] Master 127.27.40.190:36523 requested a full tablet report, sending...
W20260809 03:34:55.832021 14511 heartbeater.cc:644] Failed to heartbeat to 127.27.40.190:36523 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 03:34:55.832108 14511 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:55.832185 14511 heartbeater.cc:511] Master 127.27.40.190:36523 requested a full tablet report, sending...
I20260809 03:34:55.832612 14511 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:55.832707 14511 heartbeater.cc:511] Master 127.27.40.190:36523 requested a full tablet report, sending...
W20260809 03:34:55.833077 14511 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:34:55.933370 14517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:55.933663 14517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:55.933712 14517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:55.935231 14517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:55.935297 14517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:55.935340 14517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:34:55.937595 14517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.40.148:54217/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:44693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=33423
--tserver_master_addrs=127.27.40.190:36523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:34:55.938103 14517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:55.938393 14517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:55.949502 14529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:55.949494 14523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:55.949468 14525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:55.949922 14517 server_base.cc:1034] running on GCE node
I20260809 03:34:55.950245 14517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:55.950557 14517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:55.951756 14517 hybrid_clock.cc:648] HybridClock initialized: now 1786246495951734 us; error 35 us; skew 500 ppm
Aug 09 03:34:55 dist-test-slave-gkw7 krb5kdc[12787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246495, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:55.955178 14517 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:34:55.955662 14517 webserver.cc:460] Webserver started at http://127.27.40.130:33423/ using document root <none> and password file <none>
I20260809 03:34:55.955987 14517 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:55.956079 14517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 03:34:55 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246495, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
I20260809 03:34:55.983373 14517 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260809 03:34:55.984622 14542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:55.984853 14517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:34:55.984984 14517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "039bca5c06cf4c5691e6a0b2afbbeb36"
format_stamp: "Formatted at 2026-08-09 03:34:54 on dist-test-slave-gkw7"
server_key: "5897e727244a1dd532fac679cd338120"
server_key_iv: "b2f01cb1880ee3c99ddfcb6d6d332807"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:55.985492 14517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:56.024597 14517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:56.027102 14517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:56.027284 14517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:56.027563 14517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:34:56.027974 14517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:34:56.028030 14517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:56.028065 14517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:34:56.028083 14517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:56.035836 14517 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:44693
I20260809 03:34:56.035939 14712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:44693 every 8 connection(s)
I20260809 03:34:56.036351 14517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
Aug 09 03:34:56 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246495, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:34:56.044759 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 14517
I20260809 03:34:56.045068 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.40.148:54217/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:44641
--local_ip_for_outbound_sockets=127.27.40.131
--tserver_master_addrs=127.27.40.190:36523
--webserver_port=42149
--webserver_interface=127.27.40.131
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.40.148:54217/kms with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:34:56.047333 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:56.037181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:44441 (local address 127.27.40.190:36523)
0809 03:34:56.037250 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:56.037253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:56.037460 (+ 207us) server_negotiation.cc:407] Connection header received
0809 03:34:56.038098 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:56.038100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:56.038112 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:56.038155 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:56.038829 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.039409 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.040518 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.040838 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.040933 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:56.043617 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:56.043619 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:56.043619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:56.043628 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:56.045488 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:56.046077 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:56.046079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:56.046079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:56.046141 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:56.046476 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:56.046477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:56.046478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:56.046651 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:56.046727 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:56.047133 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 03:34:56.047240 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 03:34:56.047950 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:56.037146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:36523 (local address 127.27.40.130:44441)
0809 03:34:56.037330 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:56.037375 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:56.037985 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:56.038263 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:56.038266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:56.038458 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:34:56.038713 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:56.038719 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.039504 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.039506 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:56.040414 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:56.040417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.040978 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.040980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:56.041142 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:56.041837 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:34:56.041859 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:34:56.043498 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:34:56.045638 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:56.045646 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:56.045649 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:56.045978 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:56.046240 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:56.046240 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:56.046241 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:56.046383 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:56.046748 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:34:56.046755 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:34:56.046930 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 03:34:56.047518 (+ 588us) client_negotiation.cc:240] Negotiation successful
0809 03:34:56.047702 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260809 03:34:56.048532 14713 heartbeater.cc:347] Connected to a master server at 127.27.40.190:36523
I20260809 03:34:56.048648 14713 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:56.048862 14713 heartbeater.cc:511] Master 127.27.40.190:36523 requested a full tablet report, sending...
W20260809 03:34:56.049678 14713 heartbeater.cc:644] Failed to heartbeat to 127.27.40.190:36523 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 03:34:56.049773 14713 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:56.049857 14713 heartbeater.cc:511] Master 127.27.40.190:36523 requested a full tablet report, sending...
I20260809 03:34:56.050312 14713 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:56.050401 14713 heartbeater.cc:511] Master 127.27.40.190:36523 requested a full tablet report, sending...
W20260809 03:34:56.050735 14713 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:34:56.148682 14717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:56.148984 14717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:56.149035 14717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:56.150741 14717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:56.150811 14717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:56.150854 14717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:34:56.153038 14717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.40.148:54217/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:44641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=42149
--tserver_master_addrs=127.27.40.190:36523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:34:56.153312 14717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:56.153719 14717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:56.160854 14735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:56.161195 14740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:56.161810 14736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:56.162067 14717 server_base.cc:1034] running on GCE node
I20260809 03:34:56.162364 14717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:56.162686 14717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:56.163870 14717 hybrid_clock.cc:648] HybridClock initialized: now 1786246496163861 us; error 33 us; skew 500 ppm
Aug 09 03:34:56 dist-test-slave-gkw7 krb5kdc[12787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246496, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:56.166821 14717 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:34:56.167359 14717 webserver.cc:460] Webserver started at http://127.27.40.131:42149/ using document root <none> and password file <none>
I20260809 03:34:56.167635 14717 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:56.167732 14717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 03:34:56 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246496, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
I20260809 03:34:56.187026 14717 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260809 03:34:56.188313 14754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:56.188500 14717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:34:56.188596 14717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "ca7e1e468c72443698f8e7baf8cafbc6"
format_stamp: "Formatted at 2026-08-09 03:34:55 on dist-test-slave-gkw7"
server_key: "3c701ba368da7cdca17cdfcee4116016"
server_key_iv: "b910ccf81580e4ea5a921ac42dd83717"
server_key_version: "kuduclusterkey@0"
I20260809 03:34:56.189114 14717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:56.226467 14717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:56.229265 14717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:56.229462 14717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:56.229796 14717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:34:56.230211 14717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:34:56.230269 14717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:56.230310 14717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:34:56.230345 14717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:56.247668 14717 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:44641
I20260809 03:34:56.248220 14717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:34:56.251362 14910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:44641 every 8 connection(s)
I20260809 03:34:56.256527 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 14717
Aug 09 03:34:56 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246496, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:34:56.262570 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:56.249707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:42263 (local address 127.27.40.190:36523)
0809 03:34:56.249763 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:56.249765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:56.252098 (+ 2333us) server_negotiation.cc:407] Connection header received
0809 03:34:56.252845 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:56.252846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:56.252862 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:56.252896 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:56.253822 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.254332 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.255391 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.255639 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.255700 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:56.258834 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:56.258835 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:56.258836 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:56.258845 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:56.260693 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:56.261235 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:56.261236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:56.261237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:56.261315 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:56.261682 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:56.261684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:56.261685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:56.261860 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:56.261925 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:56.262341 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 03:34:56.262464 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 03:34:56.262879 14915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:56.249555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:36523 (local address 127.27.40.131:42263)
0809 03:34:56.251944 (+ 2389us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:56.251991 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:56.252705 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:56.253100 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:56.253104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:56.253361 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:34:56.253703 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:56.253715 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.254442 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.254443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:56.255279 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:56.255281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.255803 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.255805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:56.255958 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:56.256879 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:34:56.256891 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:34:56.258720 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:34:56.260778 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:56.260785 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:56.260788 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:56.261117 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:56.261402 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:56.261404 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:56.261404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:56.261588 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:56.261967 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:34:56.261974 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:34:56.262158 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 03:34:56.262437 (+ 279us) client_negotiation.cc:240] Negotiation successful
0809 03:34:56.262613 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2345,"thread_start_us":2198,"threads_started":1}
I20260809 03:34:56.263435 14911 heartbeater.cc:347] Connected to a master server at 127.27.40.190:36523
I20260809 03:34:56.263547 14911 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:56.263782 14911 heartbeater.cc:511] Master 127.27.40.190:36523 requested a full tablet report, sending...
W20260809 03:34:56.264604 14911 heartbeater.cc:644] Failed to heartbeat to 127.27.40.190:36523 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 03:34:56.264683 14911 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:56.264755 14911 heartbeater.cc:511] Master 127.27.40.190:36523 requested a full tablet report, sending...
I20260809 03:34:56.265094 14911 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:56.265188 14911 heartbeater.cc:511] Master 127.27.40.190:36523 requested a full tablet report, sending...
W20260809 03:34:56.265511 14911 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 03:34:56.266026 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:56.257107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49678 (local address 127.27.40.190:36523)
0809 03:34:56.257320 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:56.257321 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:56.257522 (+ 201us) server_negotiation.cc:407] Connection header received
0809 03:34:56.257612 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:56.257613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:56.257629 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:56.257707 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:56.258520 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.259327 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.260556 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.260870 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.260955 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:56.262292 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:56.262294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:56.262294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:56.262301 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:56.264551 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:56.265119 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:56.265120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:56.265121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:56.265182 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:56.265419 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:56.265420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:56.265420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:56.265572 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:56.265738 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:56.265827 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 03:34:56.265911 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":150,"threads_started":1}
Aug 09 03:34:56 dist-test-slave-gkw7 krb5kdc[12787](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786246496, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:56.833292 14511 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:56.833493 14511 heartbeater.cc:511] Master 127.27.40.190:36523 requested a full tablet report, sending...
I20260809 03:34:56.833959 14231 ts_manager.cc:194] Registered new tserver with Master: 3e155abe61fd4d92ad0ce676f1905cdd (127.27.40.129:39741)
I20260809 03:34:56.834690 14231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:47087
I20260809 03:34:57.050911 14713 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:57.051069 14713 heartbeater.cc:511] Master 127.27.40.190:36523 requested a full tablet report, sending...
I20260809 03:34:57.051460 14231 ts_manager.cc:194] Registered new tserver with Master: 039bca5c06cf4c5691e6a0b2afbbeb36 (127.27.40.130:44693)
I20260809 03:34:57.051877 14231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:44441
Aug 09 03:34:57 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246496, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
I20260809 03:34:57.265703 14911 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:57.265885 14911 heartbeater.cc:511] Master 127.27.40.190:36523 requested a full tablet report, sending...
I20260809 03:34:57.266245 14231 ts_manager.cc:194] Registered new tserver with Master: ca7e1e468c72443698f8e7baf8cafbc6 (127.27.40.131:44641)
I20260809 03:34:57.266673 14231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:42263
I20260809 03:34:57.279369 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:34:57.293443 14946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.282032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49680 (local address 127.27.40.190:36523)
0809 03:34:57.282193 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:57.282196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:57.285750 (+ 3554us) server_negotiation.cc:407] Connection header received
0809 03:34:57.285808 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:57.285809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:57.285824 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:57.285913 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:57.286574 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.287312 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.288345 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.288641 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.288731 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:57.290306 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:57.290308 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:57.290308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:57.290317 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:57.292096 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:57.292513 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:57.292514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:57.292515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:57.292586 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:57.292818 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:57.292819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:57.292819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:57.292963 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:57.293069 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:57.293211 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 03:34:57.293334 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":78,"threads_started":1}
I20260809 03:34:57.295956 14231 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:34:57.297720 14231 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:34:57.306699 14963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.301317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:39741 (local address 127.0.0.1:44150)
0809 03:34:57.302374 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:57.302384 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:57.302428 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:57.302731 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:57.302731 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:57.302733 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:34:57.302771 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:57.302772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.304011 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.304012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:57.305299 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:57.305302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.306369 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.306370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:57.306450 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:57.306464 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 03:34:57.306542 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 03:34:57.306598 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1041,"mutex_wait_us":33,"thread_start_us":299,"threads_started":1}
I20260809 03:34:57.306872 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.300667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:44693 (local address 127.0.0.1:41056)
0809 03:34:57.301717 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:57.301728 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:57.301802 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:57.302203 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:57.302204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:57.302206 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:34:57.302246 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:57.302248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.303819 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.303820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:57.305299 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:57.305302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.306634 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.306636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:57.306701 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:57.306705 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:34:57.306755 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 03:34:57.306800 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1034,"mutex_wait_us":80,"thread_start_us":220,"threads_started":1}
I20260809 03:34:57.307026 14955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.300647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:44641 (local address 127.0.0.1:45472)
0809 03:34:57.300838 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:57.300895 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:57.300970 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:57.301893 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:57.301896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:57.301898 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:34:57.302008 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:57.302010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.303870 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.303871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:57.305353 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:57.305357 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.306476 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.306477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:57.306554 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:57.306558 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:34:57.306609 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 03:34:57.306682 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":2,"thread_start_us":102,"threads_started":1}
I20260809 03:34:57.307117 14962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.301034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41056 (local address 127.27.40.130:44693)
0809 03:34:57.301266 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:57.301271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:57.301935 (+ 664us) server_negotiation.cc:407] Connection header received
0809 03:34:57.301973 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:57.301976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:57.301993 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:57.302063 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:34:57.302332 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.303728 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.305402 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.306486 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.306550 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:57.306602 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:57.306984 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 03:34:57.307028 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":178,"threads_started":1}
I20260809 03:34:57.307526 14960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.300950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44150 (local address 127.27.40.129:39741)
0809 03:34:57.301947 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:57.301952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:57.302483 (+ 531us) server_negotiation.cc:407] Connection header received
0809 03:34:57.302520 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:57.302522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:57.302537 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:57.302606 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:34:57.302849 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.303929 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.305401 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.306206 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.306273 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:57.306332 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:57.307404 (+ 1072us) server_negotiation.cc:299] Negotiation successful
0809 03:34:57.307451 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":965,"thread_start_us":59,"threads_started":1}
I20260809 03:34:57.307850 14618 tablet_service.cc:1467] Processing CreateTablet for tablet a2fffc68b26848b7897e71c76aca41fd (DEFAULT_TABLE table=test-table [id=8839ce40646c402984f651b94e9d1134]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:34:57.308090 14397 tablet_service.cc:1467] Processing CreateTablet for tablet a2fffc68b26848b7897e71c76aca41fd (DEFAULT_TABLE table=test-table [id=8839ce40646c402984f651b94e9d1134]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:34:57.308269 14618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2fffc68b26848b7897e71c76aca41fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:34:57.308416 14397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2fffc68b26848b7897e71c76aca41fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:34:57.308384 14957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.300920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45472 (local address 127.27.40.131:44641)
0809 03:34:57.301643 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:57.301647 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:57.301659 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:34:57.301696 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:57.301699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:57.301715 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:57.301821 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:34:57.302099 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.303740 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.305527 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.306362 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.306753 (+ 391us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:57.307557 (+ 804us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:57.307930 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 03:34:57.308087 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":72,"threads_started":1}
I20260809 03:34:57.309700 14814 tablet_service.cc:1467] Processing CreateTablet for tablet a2fffc68b26848b7897e71c76aca41fd (DEFAULT_TABLE table=test-table [id=8839ce40646c402984f651b94e9d1134]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:34:57.310312 14814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2fffc68b26848b7897e71c76aca41fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:34:57.317291 14966 tablet_bootstrap.cc:492] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6: Bootstrap starting.
I20260809 03:34:57.317809 14964 tablet_bootstrap.cc:492] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36: Bootstrap starting.
I20260809 03:34:57.318516 14966 tablet_bootstrap.cc:654] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6: Neither blocks nor log segments found. Creating new log.
I20260809 03:34:57.318693 14965 tablet_bootstrap.cc:492] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd: Bootstrap starting.
I20260809 03:34:57.319329 14966 log.cc:824] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6: Log is configured to *not* fsync() on all Append() calls
I20260809 03:34:57.320459 14964 tablet_bootstrap.cc:654] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36: Neither blocks nor log segments found. Creating new log.
I20260809 03:34:57.321219 14964 log.cc:824] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36: Log is configured to *not* fsync() on all Append() calls
I20260809 03:34:57.322270 14964 tablet_bootstrap.cc:492] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36: No bootstrap required, opened a new log
I20260809 03:34:57.322299 14965 tablet_bootstrap.cc:654] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd: Neither blocks nor log segments found. Creating new log.
I20260809 03:34:57.322417 14964 ts_tablet_manager.cc:1397] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260809 03:34:57.323145 14965 log.cc:824] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd: Log is configured to *not* fsync() on all Append() calls
I20260809 03:34:57.324322 14965 tablet_bootstrap.cc:492] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd: No bootstrap required, opened a new log
I20260809 03:34:57.324419 14965 ts_tablet_manager.cc:1397] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260809 03:34:57.327764 14964 raft_consensus.cc:348] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7e1e468c72443698f8e7baf8cafbc6" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 44641 } } peers { permanent_uuid: "3e155abe61fd4d92ad0ce676f1905cdd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39741 } } peers { permanent_uuid: "039bca5c06cf4c5691e6a0b2afbbeb36" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44693 } }
I20260809 03:34:57.328045 14964 raft_consensus.cc:374] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:34:57.328130 14964 raft_consensus.cc:729] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039bca5c06cf4c5691e6a0b2afbbeb36, State: Initialized, Role: FOLLOWER
I20260809 03:34:57.328495 14964 consensus_queue.cc:260] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7e1e468c72443698f8e7baf8cafbc6" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 44641 } } peers { permanent_uuid: "3e155abe61fd4d92ad0ce676f1905cdd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39741 } } peers { permanent_uuid: "039bca5c06cf4c5691e6a0b2afbbeb36" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44693 } }
I20260809 03:34:57.328969 14964 ts_tablet_manager.cc:1428] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260809 03:34:57.329159 14713 heartbeater.cc:503] Master 127.27.40.190:36523 was elected leader, sending a full tablet report...
I20260809 03:34:57.329367 14965 raft_consensus.cc:348] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7e1e468c72443698f8e7baf8cafbc6" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 44641 } } peers { permanent_uuid: "3e155abe61fd4d92ad0ce676f1905cdd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39741 } } peers { permanent_uuid: "039bca5c06cf4c5691e6a0b2afbbeb36" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44693 } }
I20260809 03:34:57.329608 14965 raft_consensus.cc:374] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:34:57.329680 14965 raft_consensus.cc:729] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e155abe61fd4d92ad0ce676f1905cdd, State: Initialized, Role: FOLLOWER
I20260809 03:34:57.329831 14965 consensus_queue.cc:260] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7e1e468c72443698f8e7baf8cafbc6" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 44641 } } peers { permanent_uuid: "3e155abe61fd4d92ad0ce676f1905cdd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39741 } } peers { permanent_uuid: "039bca5c06cf4c5691e6a0b2afbbeb36" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44693 } }
I20260809 03:34:57.330192 14965 ts_tablet_manager.cc:1428] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 03:34:57.330334 14511 heartbeater.cc:503] Master 127.27.40.190:36523 was elected leader, sending a full tablet report...
I20260809 03:34:57.334740 14966 tablet_bootstrap.cc:492] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6: No bootstrap required, opened a new log
I20260809 03:34:57.334859 14966 ts_tablet_manager.cc:1397] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20260809 03:34:57.338521 14966 raft_consensus.cc:348] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7e1e468c72443698f8e7baf8cafbc6" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 44641 } } peers { permanent_uuid: "3e155abe61fd4d92ad0ce676f1905cdd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39741 } } peers { permanent_uuid: "039bca5c06cf4c5691e6a0b2afbbeb36" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44693 } }
I20260809 03:34:57.338732 14966 raft_consensus.cc:374] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:34:57.338785 14966 raft_consensus.cc:729] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca7e1e468c72443698f8e7baf8cafbc6, State: Initialized, Role: FOLLOWER
I20260809 03:34:57.338971 14966 consensus_queue.cc:260] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7e1e468c72443698f8e7baf8cafbc6" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 44641 } } peers { permanent_uuid: "3e155abe61fd4d92ad0ce676f1905cdd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39741 } } peers { permanent_uuid: "039bca5c06cf4c5691e6a0b2afbbeb36" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44693 } }
I20260809 03:34:57.339354 14966 ts_tablet_manager.cc:1428] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260809 03:34:57.339605 14911 heartbeater.cc:503] Master 127.27.40.190:36523 was elected leader, sending a full tablet report...
Aug 09 03:34:57 dist-test-slave-gkw7 krb5kdc[12787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246496, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for HTTP/127.27.40.148@KRBTEST.COM
W20260809 03:34:57.502111 14913 tablet.cc:2369] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:34:57.537766 14715 tablet.cc:2369] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:34:57.561362 14971 raft_consensus.cc:482] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:34:57.561607 14971 raft_consensus.cc:504] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7e1e468c72443698f8e7baf8cafbc6" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 44641 } } peers { permanent_uuid: "3e155abe61fd4d92ad0ce676f1905cdd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39741 } } peers { permanent_uuid: "039bca5c06cf4c5691e6a0b2afbbeb36" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44693 } }
I20260809 03:34:57.562245 14971 leader_election.cc:288] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca7e1e468c72443698f8e7baf8cafbc6 (127.27.40.131:44641), 039bca5c06cf4c5691e6a0b2afbbeb36 (127.27.40.130:44693)
W20260809 03:34:57.568213 14512 tablet.cc:2369] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:34:57.575160 14957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.562774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:55297 (local address 127.27.40.131:44641)
0809 03:34:57.562844 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:57.562846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:57.568691 (+ 5845us) server_negotiation.cc:407] Connection header received
0809 03:34:57.568834 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:57.568836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:57.568847 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:57.568880 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:34:57.569242 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.570009 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.573897 (+ 3888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.574551 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.574624 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:57.574637 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:57.575017 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 03:34:57.575069 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 03:34:57.575544 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.562428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:44641 (local address 127.27.40.129:55297)
0809 03:34:57.568573 (+ 6145us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:57.568591 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:57.568747 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:57.568954 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:57.568955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:57.568957 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:34:57.569147 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:57.569149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.570132 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.570133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:57.571592 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:57.571595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.574816 (+ 3221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.574817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:57.574924 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:57.574929 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:34:57.575299 (+ 370us) client_negotiation.cc:240] Negotiation successful
0809 03:34:57.575347 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6093,"thread_start_us":168,"threads_started":1}
I20260809 03:34:57.576189 14827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2fffc68b26848b7897e71c76aca41fd" candidate_uuid: "3e155abe61fd4d92ad0ce676f1905cdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca7e1e468c72443698f8e7baf8cafbc6" is_pre_election: true
I20260809 03:34:57.576365 14827 raft_consensus.cc:2454] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e155abe61fd4d92ad0ce676f1905cdd in term 0.
I20260809 03:34:57.576778 14356 leader_election.cc:302] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e155abe61fd4d92ad0ce676f1905cdd, ca7e1e468c72443698f8e7baf8cafbc6; no voters:
I20260809 03:34:57.576920 14971 raft_consensus.cc:2781] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:34:57.576960 14971 raft_consensus.cc:482] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:34:57.576982 14971 raft_consensus.cc:3037] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:34:57.578027 14971 raft_consensus.cc:504] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7e1e468c72443698f8e7baf8cafbc6" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 44641 } } peers { permanent_uuid: "3e155abe61fd4d92ad0ce676f1905cdd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39741 } } peers { permanent_uuid: "039bca5c06cf4c5691e6a0b2afbbeb36" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44693 } }
I20260809 03:34:57.578190 14971 leader_election.cc:288] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [CANDIDATE]: Term 1 election: Requested vote from peers ca7e1e468c72443698f8e7baf8cafbc6 (127.27.40.131:44641), 039bca5c06cf4c5691e6a0b2afbbeb36 (127.27.40.130:44693)
I20260809 03:34:57.578645 14827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2fffc68b26848b7897e71c76aca41fd" candidate_uuid: "3e155abe61fd4d92ad0ce676f1905cdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca7e1e468c72443698f8e7baf8cafbc6"
I20260809 03:34:57.578727 14827 raft_consensus.cc:3037] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:34:57.579607 14827 raft_consensus.cc:2454] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e155abe61fd4d92ad0ce676f1905cdd in term 1.
I20260809 03:34:57.579841 14356 leader_election.cc:302] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e155abe61fd4d92ad0ce676f1905cdd, ca7e1e468c72443698f8e7baf8cafbc6; no voters:
I20260809 03:34:57.579965 14971 raft_consensus.cc:2781] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:34:57.580171 14971 raft_consensus.cc:686] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [term 1 LEADER]: Becoming Leader. State: Replica: 3e155abe61fd4d92ad0ce676f1905cdd, State: Running, Role: LEADER
I20260809 03:34:57.580278 14971 consensus_queue.cc:237] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7e1e468c72443698f8e7baf8cafbc6" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 44641 } } peers { permanent_uuid: "3e155abe61fd4d92ad0ce676f1905cdd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39741 } } peers { permanent_uuid: "039bca5c06cf4c5691e6a0b2afbbeb36" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44693 } }
I20260809 03:34:57.581171 14231 catalog_manager.cc:5495] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e155abe61fd4d92ad0ce676f1905cdd (127.27.40.129). New cstate: current_term: 1 leader_uuid: "3e155abe61fd4d92ad0ce676f1905cdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7e1e468c72443698f8e7baf8cafbc6" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 44641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e155abe61fd4d92ad0ce676f1905cdd" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 39741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039bca5c06cf4c5691e6a0b2afbbeb36" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44693 } health_report { overall_health: UNKNOWN } } }
I20260809 03:34:57.583249 14962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.562688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:33137 (local address 127.27.40.130:44693)
0809 03:34:57.562739 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:57.562753 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:57.574692 (+ 11939us) server_negotiation.cc:407] Connection header received
0809 03:34:57.574853 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:57.574854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:57.574865 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:57.574915 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:34:57.575144 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.575870 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.582227 (+ 6357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.582792 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.582846 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:57.582856 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:57.583085 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 03:34:57.583182 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 03:34:57.583626 14979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.562600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:44693 (local address 127.27.40.129:33137)
0809 03:34:57.574612 (+ 12012us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:57.574625 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:57.574723 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:57.574987 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:57.574988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:57.574990 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:34:57.575057 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:57.575058 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.580908 (+ 5850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.580909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:57.582101 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:57.582104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.582921 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.582922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:57.582996 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:57.583000 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:34:57.583496 (+ 496us) client_negotiation.cc:240] Negotiation successful
0809 03:34:57.583541 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11986,"thread_start_us":182,"threads_started":1}
I20260809 03:34:57.584136 14644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2fffc68b26848b7897e71c76aca41fd" candidate_uuid: "3e155abe61fd4d92ad0ce676f1905cdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039bca5c06cf4c5691e6a0b2afbbeb36" is_pre_election: true
I20260809 03:34:57.584280 14644 raft_consensus.cc:2454] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e155abe61fd4d92ad0ce676f1905cdd in term 0.
I20260809 03:34:57.584398 14643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2fffc68b26848b7897e71c76aca41fd" candidate_uuid: "3e155abe61fd4d92ad0ce676f1905cdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039bca5c06cf4c5691e6a0b2afbbeb36"
I20260809 03:34:57.584457 14643 raft_consensus.cc:3037] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:34:57.585348 14643 raft_consensus.cc:2454] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e155abe61fd4d92ad0ce676f1905cdd in term 1.
I20260809 03:34:57.591634 14960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.588280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44154 (local address 127.27.40.129:39741)
0809 03:34:57.588417 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:57.588418 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:57.588425 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:34:57.588455 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:57.588456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:57.588474 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:57.588532 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:34:57.588708 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.589435 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.590595 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.590882 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.590946 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:57.591153 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:34:57.591270 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:34:57.591339 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:57.591546 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 03:34:57.591568 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260809 03:34:57.593631 14643 raft_consensus.cc:1261] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36 [term 1 FOLLOWER]: Refusing update from remote peer 3e155abe61fd4d92ad0ce676f1905cdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:34:57.594080 14980 consensus_queue.cc:1035] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "039bca5c06cf4c5691e6a0b2afbbeb36" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:34:57.594498 14827 raft_consensus.cc:1261] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3e155abe61fd4d92ad0ce676f1905cdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:34:57.596061 14980 consensus_queue.cc:1035] T a2fffc68b26848b7897e71c76aca41fd P 3e155abe61fd4d92ad0ce676f1905cdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca7e1e468c72443698f8e7baf8cafbc6" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 44641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:34:57.599020 14985 mvcc.cc:205] Tried to move back new op lower bound from 7316465654140829696 to 7316465654089351168. Current Snapshot: MvccSnapshot[applied={T|T < 7316465654140829696}]
I20260809 03:34:57.599701 14990 mvcc.cc:205] Tried to move back new op lower bound from 7316465654140829696 to 7316465654089351168. Current Snapshot: MvccSnapshot[applied={T|T < 7316465654140829696}]
I20260809 03:34:57.603773 14986 mvcc.cc:205] Tried to move back new op lower bound from 7316465654140829696 to 7316465654089351168. Current Snapshot: MvccSnapshot[applied={T|T < 7316465654140829696}]
I20260809 03:34:57.607986 14230 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:34:57.608120 14230 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:34:57.608994 14230 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 787bb3590fef49d396d6b21187377426: Sending DeleteTablet for 3 replicas of tablet a2fffc68b26848b7897e71c76aca41fd
I20260809 03:34:57.609786 14814 tablet_service.cc:1514] Processing DeleteTablet for tablet a2fffc68b26848b7897e71c76aca41fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:34:57 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:45472
I20260809 03:34:57.610357 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet a2fffc68b26848b7897e71c76aca41fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:34:57 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:44150
I20260809 03:34:57.610950 14618 tablet_service.cc:1514] Processing DeleteTablet for tablet a2fffc68b26848b7897e71c76aca41fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:34:57 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:41056
I20260809 03:34:57.611431 15001 tablet_replica.cc:330] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36: stopping tablet replica
I20260809 03:34:57.611630 15001 raft_consensus.cc:2229] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:34:57.611845 15001 raft_consensus.cc:2258] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:34:57.611884 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 14284
I20260809 03:34:57.615073 15001 ts_tablet_manager.cc:1905] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:34:57.618039 15001 ts_tablet_manager.cc:1918] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:34:57.618131 15001 log.cc:1196] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/a2fffc68b26848b7897e71c76aca41fd
I20260809 03:34:57.618464 15001 ts_tablet_manager.cc:1939] T a2fffc68b26848b7897e71c76aca41fd P 039bca5c06cf4c5691e6a0b2afbbeb36: Deleting consensus metadata
I20260809 03:34:57.619014 14211 catalog_manager.cc:4841] TS 039bca5c06cf4c5691e6a0b2afbbeb36 (127.27.40.130:44693): tablet a2fffc68b26848b7897e71c76aca41fd (table test-table [id=8839ce40646c402984f651b94e9d1134]) successfully deleted
I20260809 03:34:57.628533 14998 tablet_replica.cc:330] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6: stopping tablet replica
I20260809 03:34:57.628705 14998 raft_consensus.cc:2229] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:34:57.628845 14998 raft_consensus.cc:2258] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 03:34:57.634251 14214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.40.129:39741 (error 108)
I20260809 03:34:57.635270 14955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.635143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:39741 (local address 127.0.0.1:44166)
0809 03:34:57.635203 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:57.635235 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:39741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 03:34:57.635387 14214 catalog_manager.cc:4584] TS 3e155abe61fd4d92ad0ce676f1905cdd (127.27.40.129:39741): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a2fffc68b26848b7897e71c76aca41fd: Network error: Client connection negotiation failed: client connection to 127.27.40.129:39741: connect: Connection refused (error 111)
I20260809 03:34:57.635583 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 14517
I20260809 03:34:57.642660 14998 ts_tablet_manager.cc:1905] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:34:57.644567 14998 ts_tablet_manager.cc:1918] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:34:57.644661 14998 log.cc:1196] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/a2fffc68b26848b7897e71c76aca41fd
I20260809 03:34:57.644904 14998 ts_tablet_manager.cc:1939] T a2fffc68b26848b7897e71c76aca41fd P ca7e1e468c72443698f8e7baf8cafbc6: Deleting consensus metadata
I20260809 03:34:57.645347 14213 catalog_manager.cc:4841] TS ca7e1e468c72443698f8e7baf8cafbc6 (127.27.40.131:44641): tablet a2fffc68b26848b7897e71c76aca41fd (table test-table [id=8839ce40646c402984f651b94e9d1134]) successfully deleted
I20260809 03:34:57.670228 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 14717
I20260809 03:34:57.688759 14955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.688621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:39741 (local address 127.0.0.1:44180)
0809 03:34:57.688678 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:57.688706 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:39741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 03:34:57.689035 14955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.688975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:39741 (local address 127.0.0.1:44182)
0809 03:34:57.688999 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:57.689015 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:39741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260809 03:34:57.695993 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 14185
I20260809 03:34:57.708667 27810 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 03:34:58.054200 27810 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 03:34:58.054301 27810 mini_ranger.cc:66] Stopping Ranger...
I20260809 03:34:58.409595 27810 mini_ranger.cc:68] Stopped Ranger
2026-08-09 03:34:58.409 UTC [12832] LOG: received smart shutdown request
2026-08-09 03:34:58.414 UTC [12832] LOG: background worker "logical replication launcher" (PID 12842) exited with exit code 1
2026-08-09 03:34:58.415 UTC [12836] LOG: shutting down
2026-08-09 03:34:58.605 UTC [12832] LOG: database system is shut down
2026-08-09T03:34:58Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48927 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:34:58 dist-test-slave-gkw7 krb5kdc[15014](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:34:58 dist-test-slave-gkw7 krb5kdc[15014](info): set up 2 sockets
Aug 09 03:34:58 dist-test-slave-gkw7 krb5kdc[15014](info): commencing operation
krb5kdc: starting...
W20260809 03:34:59.959684 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:35:00 dist-test-slave-gkw7 krb5kdc[15014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:35:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:35:00.163820 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:35165
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:35165,127.27.40.189:43649,127.27.40.188:46179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:00.347360 15037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:00.347973 15037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:00.348066 15037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:00.350956 15037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:00.351064 15037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:00.351111 15037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:00.351267 15037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:00.351414 15037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:00.355166 15037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:35165,127.27.40.189:43649,127.27.40.188:46179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:35165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:00.355737 15037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:00.356210 15037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:00.364668 15045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:00.364656 15046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:00.364658 15051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:00.365391 15037 server_base.cc:1034] running on GCE node
I20260809 03:35:00.365849 15037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:00.366487 15037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:00.367753 15037 hybrid_clock.cc:648] HybridClock initialized: now 1786246500367723 us; error 44 us; skew 500 ppm
Aug 09 03:35:00 dist-test-slave-gkw7 krb5kdc[15014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:00.371910 15037 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:35:00.372665 15037 webserver.cc:460] Webserver started at http://127.27.40.190:32863/ using document root <none> and password file <none>
I20260809 03:35:00.373019 15037 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:00.373172 15037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:00.373425 15037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:00.375200 15037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9"
format_stamp: "Formatted at 2026-08-09 03:35:00 on dist-test-slave-gkw7"
server_key: "84b3372ddf25f1154b29a2d0dc30afaf"
server_key_iv: "f6cc4a12cef549514ebdd70d1fd6b1c0"
server_key_version: "encryptionkey@0"
I20260809 03:35:00.375839 15037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9"
format_stamp: "Formatted at 2026-08-09 03:35:00 on dist-test-slave-gkw7"
server_key: "84b3372ddf25f1154b29a2d0dc30afaf"
server_key_iv: "f6cc4a12cef549514ebdd70d1fd6b1c0"
server_key_version: "encryptionkey@0"
I20260809 03:35:00.378335 15037 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260809 03:35:00.379539 15061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:00.379798 15037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:00.379978 15037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9"
format_stamp: "Formatted at 2026-08-09 03:35:00 on dist-test-slave-gkw7"
server_key: "84b3372ddf25f1154b29a2d0dc30afaf"
server_key_iv: "f6cc4a12cef549514ebdd70d1fd6b1c0"
server_key_version: "encryptionkey@0"
I20260809 03:35:00.380105 15037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:00.405256 15037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:00.408494 15037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:00.408802 15037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:00.416182 15037 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:35165
I20260809 03:35:00.416276 15135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:35165 every 8 connection(s)
I20260809 03:35:00.416898 15037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:00.418460 15136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:00.418814 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 15037
I20260809 03:35:00.419008 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:35:00.419440 27810 external_mini_cluster.cc:1428] Setting key ae991d07f50fdb3f610388faf61a8585
I20260809 03:35:00.423097 15136 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.40.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.189@KRBTEST.COM" created.
Aug 09 03:35:00 dist-test-slave-gkw7 krb5kdc[15014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:00.443713 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.424360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52646 (local address 127.27.40.190:35165)
0809 03:35:00.424613 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.424644 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.424881 (+ 237us) server_negotiation.cc:407] Connection header received
0809 03:35:00.425865 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.425875 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.425904 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.426091 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:00.427247 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.428516 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.430105 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.430608 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.430865 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.437790 (+ 6925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:00.437799 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:00.437799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:00.437807 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:00.440740 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.441460 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.441470 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.441471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.441672 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.442197 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.442198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.442199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.442530 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:00.442717 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.443010 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.443216 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":134,"threads_started":1}
I20260809 03:35:00.443713 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.424360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:35165 (local address 127.0.0.1:52646)
0809 03:35:00.424683 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.424736 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:00.425509 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:00.426106 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:00.426111 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:00.426754 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:00.427068 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.427075 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.428721 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.428730 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.429935 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.429938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.430797 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.430799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.430917 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.431746 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:00.431756 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:00.437606 (+ 5850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:00.440948 (+ 3342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:00.440954 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:00.440956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.441268 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.441844 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:00.441845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:00.441845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.442053 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.442670 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:00.442675 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:00.442790 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 03:35:00.442945 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 03:35:00.443325 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":120,"thread_start_us":149,"threads_started":1}
I20260809 03:35:00.446113 15136 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:35:00.446705 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.446500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:43649 (local address 127.0.0.1:40106)
0809 03:35:00.446586 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.446657 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:43649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 03:35:00.446921 15065 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.40.189:43649: connect: Connection refused (error 111)
I20260809 03:35:00.447576 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.447414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:43649 (local address 127.0.0.1:40110)
0809 03:35:00.447477 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.447544 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:43649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 03:35:00.447747 15136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.189:43649: Network error: Client connection negotiation failed: client connection to 127.27.40.189:43649: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.189.keytab.
Entry for principal kudu/127.27.40.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.189.keytab.
I20260809 03:35:00.458153 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.189:43649
--webserver_interface=127.27.40.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.189.keytab
--principal=kudu/127.27.40.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:35165,127.27.40.189:43649,127.27.40.188:46179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:35:00.497088 15136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } attempt: 1
I20260809 03:35:00.497885 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.497691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:43649 (local address 127.0.0.1:40116)
0809 03:35:00.497795 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.497846 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:43649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 03:35:00.498509 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.498398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:43649 (local address 127.0.0.1:40128)
0809 03:35:00.498450 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.498492 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:43649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 03:35:00.498696 15136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.189:43649: Network error: Client connection negotiation failed: client connection to 127.27.40.189:43649: connect: Connection refused (error 111)
I20260809 03:35:00.566957 15136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } attempt: 2
I20260809 03:35:00.567752 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.567529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:43649 (local address 127.0.0.1:40136)
0809 03:35:00.567637 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.567690 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:43649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 03:35:00.568418 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.568233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:43649 (local address 127.0.0.1:40152)
0809 03:35:00.568328 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.568383 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:43649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 03:35:00.568717 15136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.189:43649: Network error: Client connection negotiation failed: client connection to 127.27.40.189:43649: connect: Connection refused (error 111)
W20260809 03:35:00.614311 15143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:00.615134 15143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:00.615278 15143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:00.619220 15143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:00.619454 15143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:00.619473 15143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:00.619488 15143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:00.619613 15143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:00.625319 15143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:35165,127.27.40.189:43649,127.27.40.188:46179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.189.keytab
--principal=kudu/127.27.40.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.189:43649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.40.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:00.626262 15143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:00.626897 15143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:00.640559 15149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:00.640519 15150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:00.640519 15152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:00.640604 15143 server_base.cc:1034] running on GCE node
I20260809 03:35:00.641197 15143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:00.641824 15143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:00.643041 15143 hybrid_clock.cc:648] HybridClock initialized: now 1786246500643016 us; error 35 us; skew 500 ppm
Aug 09 03:35:00 dist-test-slave-gkw7 krb5kdc[15014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:00.651629 15143 init.cc:377] Logged in from keytab as kudu/127.27.40.189@KRBTEST.COM (short username kudu)
I20260809 03:35:00.652890 15143 webserver.cc:460] Webserver started at http://127.27.40.189:45923/ using document root <none> and password file <none>
I20260809 03:35:00.653648 15143 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:00.653864 15143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:00.654167 15143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:00.657238 15143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/data/instance:
uuid: "f93b35768efa4e31a5d0c12d5f9178cc"
format_stamp: "Formatted at 2026-08-09 03:35:00 on dist-test-slave-gkw7"
server_key: "090db9a5a4802f962cec89bcbb5642f1"
server_key_iv: "624a63a8decc067fda35a3b9f9946470"
server_key_version: "encryptionkey@0"
I20260809 03:35:00.658420 15143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/wal/instance:
uuid: "f93b35768efa4e31a5d0c12d5f9178cc"
format_stamp: "Formatted at 2026-08-09 03:35:00 on dist-test-slave-gkw7"
server_key: "090db9a5a4802f962cec89bcbb5642f1"
server_key_iv: "624a63a8decc067fda35a3b9f9946470"
server_key_version: "encryptionkey@0"
I20260809 03:35:00.659930 15136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } attempt: 3
I20260809 03:35:00.660983 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.660485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:43649 (local address 127.0.0.1:40154)
0809 03:35:00.660619 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.660731 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:43649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 03:35:00.661878 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.661718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:43649 (local address 127.0.0.1:40162)
0809 03:35:00.661774 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.661834 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:43649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 03:35:00.662163 15136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.189:43649: Network error: Client connection negotiation failed: client connection to 127.27.40.189:43649: connect: Connection refused (error 111)
I20260809 03:35:00.663609 15143 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 03:35:00.665519 15167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:00.666217 15143 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:35:00.666549 15143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/wal
uuid: "f93b35768efa4e31a5d0c12d5f9178cc"
format_stamp: "Formatted at 2026-08-09 03:35:00 on dist-test-slave-gkw7"
server_key: "090db9a5a4802f962cec89bcbb5642f1"
server_key_iv: "624a63a8decc067fda35a3b9f9946470"
server_key_version: "encryptionkey@0"
I20260809 03:35:00.666769 15143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:00.694501 15143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:00.702384 15143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:00.702781 15143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:00.717584 15143 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.189:43649
I20260809 03:35:00.717720 15235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.189:43649 every 8 connection(s)
I20260809 03:35:00.718748 15143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/data/info.pb
I20260809 03:35:00.720983 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 15143
I20260809 03:35:00.721199 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-1/wal/instance
I20260809 03:35:00.721424 15236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:00.721796 27810 external_mini_cluster.cc:1428] Setting key 2327938f8eaa05bc06c6a396917c68db
I20260809 03:35:00.726441 15236 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:35:00 dist-test-slave-gkw7 krb5kdc[15014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.189@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.40.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.188@KRBTEST.COM" created.
I20260809 03:35:00.761732 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.728837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52654 (local address 127.27.40.190:35165)
0809 03:35:00.728998 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.729010 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.729901 (+ 891us) server_negotiation.cc:407] Connection header received
0809 03:35:00.731043 (+ 1142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.731045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.731104 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.731456 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:00.733649 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.736575 (+ 2926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.739621 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.740994 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.741329 (+ 335us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.749708 (+ 8379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:00.749710 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:00.749710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:00.749718 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:00.754901 (+ 5183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.757371 (+ 2470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.757378 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.757378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.757521 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.758753 (+ 1232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.758760 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.758761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.759143 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:00.759357 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.761092 (+ 1735us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.761237 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260809 03:35:00.762892 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.728592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:35165 (local address 127.0.0.1:52654)
0809 03:35:00.729299 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.729624 (+ 325us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:00.730833 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:00.731588 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:00.731612 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:00.732522 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:00.733268 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.733273 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.737046 (+ 3773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.737047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.739348 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.739351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.741381 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.741383 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.741909 (+ 526us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.743174 (+ 1265us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:00.743235 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:00.749426 (+ 6191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:00.755234 (+ 5808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:00.755246 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:00.755249 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.757018 (+ 1769us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.757801 (+ 783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:00.757802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:00.757802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.758558 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.759476 (+ 918us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:00.759494 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:00.760525 (+ 1031us) client_negotiation.cc:769] Sending connection context
0809 03:35:00.761046 (+ 521us) client_negotiation.cc:240] Negotiation successful
0809 03:35:00.761670 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":365,"threads_started":1}
I20260809 03:35:00.765236 15236 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:35:00 dist-test-slave-gkw7 krb5kdc[15014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.189@KRBTEST.COM for kudu/127.27.40.189@KRBTEST.COM
I20260809 03:35:00.790575 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.766074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40178 (local address 127.27.40.189:43649)
0809 03:35:00.766730 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.766734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.766794 (+ 60us) server_negotiation.cc:407] Connection header received
0809 03:35:00.766888 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.766897 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.766959 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.767344 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:00.768125 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.770427 (+ 2302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.772198 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.773039 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.773422 (+ 383us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.780266 (+ 6844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:00.780276 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:00.780276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:00.780305 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:00.785851 (+ 5546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.786966 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.787000 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.787007 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.787139 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.788008 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.788017 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.788025 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.788601 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:00.789063 (+ 462us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.789616 (+ 553us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.789891 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":510,"threads_started":1}
I20260809 03:35:00.790822 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.765965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:43649 (local address 127.0.0.1:40178)
0809 03:35:00.766067 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.766077 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:00.766224 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:00.767340 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:00.767342 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:00.767720 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:00.767903 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.767911 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.770804 (+ 2893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.770805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.771966 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.771974 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.773401 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.773402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.773740 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.775237 (+ 1497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:00.775255 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:00.779898 (+ 4643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:00.786104 (+ 6206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:00.786113 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:00.786114 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.786535 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.787403 (+ 868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:00.787404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:00.787404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.787557 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.788873 (+ 1316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:00.788874 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:00.789095 (+ 221us) client_negotiation.cc:769] Sending connection context
0809 03:35:00.789431 (+ 336us) client_negotiation.cc:240] Negotiation successful
0809 03:35:00.790202 (+ 771us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":175}
I20260809 03:35:00.792973 15236 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:35:00.794082 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.793725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:46179 (local address 127.0.0.1:55548)
0809 03:35:00.793826 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.793948 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 03:35:00.794497 15174 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.40.188:46179: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.188.keytab.
Entry for principal kudu/127.27.40.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.188.keytab.
I20260809 03:35:00.796178 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.795975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:46179 (local address 127.0.0.1:55556)
0809 03:35:00.796049 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.796095 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 03:35:00.797847 15236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.188:46179: Network error: Client connection negotiation failed: client connection to 127.27.40.188:46179: connect: Connection refused (error 111)
I20260809 03:35:00.797866 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.188:46179
--webserver_interface=127.27.40.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.188.keytab
--principal=kudu/127.27.40.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:35165,127.27.40.189:43649,127.27.40.188:46179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:35:00.805446 15136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } attempt: 4
Aug 09 03:35:00 dist-test-slave-gkw7 krb5kdc[15014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for kudu/127.27.40.189@KRBTEST.COM
I20260809 03:35:00.824771 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.806595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:43649 (local address 127.0.0.1:40180)
0809 03:35:00.806785 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.806798 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:00.806920 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:00.807358 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:00.807359 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:00.807632 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:00.807721 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.807723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.810616 (+ 2893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.810617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.811547 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.811549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.812953 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.812954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.813053 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.814226 (+ 1173us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:00.814248 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:00.816983 (+ 2735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:00.822279 (+ 5296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:00.822286 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:00.822286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.822669 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.823240 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:00.823241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:00.823241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.823363 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.824271 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:00.824272 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:00.824336 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 03:35:00.824416 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 03:35:00.824603 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 03:35:00.825381 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.806903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40180 (local address 127.27.40.189:43649)
0809 03:35:00.807127 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.807138 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.807149 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:35:00.807184 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.807193 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.807213 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.807349 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:00.808103 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.810374 (+ 2271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.811679 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.812783 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.812951 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.817107 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:00.817110 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:00.817110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:00.817136 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:00.821996 (+ 4860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.822787 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.822797 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.822798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.823003 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.823478 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.823480 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.823480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.824003 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:00.824311 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.824565 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.824892 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260809 03:35:00.826417 15136 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:35:00.827199 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.826909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:46179 (local address 127.0.0.1:55566)
0809 03:35:00.827075 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.827104 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 03:35:00.828068 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.827804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:46179 (local address 127.0.0.1:55582)
0809 03:35:00.827981 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.828036 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 03:35:00.828401 15136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.188:46179: Network error: Client connection negotiation failed: client connection to 127.27.40.188:46179: connect: Connection refused (error 111)
I20260809 03:35:00.847368 15236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } attempt: 1
I20260809 03:35:00.847887 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.847720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:46179 (local address 127.0.0.1:55588)
0809 03:35:00.847795 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.847848 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 03:35:00.848268 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.848174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:46179 (local address 127.0.0.1:55594)
0809 03:35:00.848220 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.848248 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 03:35:00.848368 15236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.188:46179: Network error: Client connection negotiation failed: client connection to 127.27.40.188:46179: connect: Connection refused (error 111)
I20260809 03:35:00.887593 15136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } attempt: 1
I20260809 03:35:00.888162 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.887956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:46179 (local address 127.0.0.1:55606)
0809 03:35:00.888049 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.888111 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 03:35:00.888808 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.888617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:46179 (local address 127.0.0.1:55616)
0809 03:35:00.888705 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.888766 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260809 03:35:00.888914 15136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.188:46179: Network error: Client connection negotiation failed: client connection to 127.27.40.188:46179: connect: Connection refused (error 111)
W20260809 03:35:00.909232 15245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:00.909427 15245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:00.909461 15245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:00.910684 15245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:00.910729 15245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:00.910744 15245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:00.910756 15245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:00.910768 15245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:00.912212 15245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:35165,127.27.40.189:43649,127.27.40.188:46179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.188.keytab
--principal=kudu/127.27.40.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.188:46179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.40.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:00.912402 15245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:00.912582 15245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:35:00.916491 15236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } attempt: 2
I20260809 03:35:00.917017 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.916788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:46179 (local address 127.0.0.1:55624)
0809 03:35:00.916893 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.916979 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 03:35:00.917547 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.917394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:46179 (local address 127.0.0.1:55626)
0809 03:35:00.917438 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.917463 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 03:35:00.917476 15257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:00.917469 15253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:00.917471 15254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:00.917704 15236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.188:46179: Network error: Client connection negotiation failed: client connection to 127.27.40.188:46179: connect: Connection refused (error 111)
I20260809 03:35:00.918040 15245 server_base.cc:1034] running on GCE node
I20260809 03:35:00.918288 15245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:00.918571 15245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:00.919741 15245 hybrid_clock.cc:648] HybridClock initialized: now 1786246500919733 us; error 45 us; skew 500 ppm
Aug 09 03:35:00 dist-test-slave-gkw7 krb5kdc[15014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:00.922216 15245 init.cc:377] Logged in from keytab as kudu/127.27.40.188@KRBTEST.COM (short username kudu)
I20260809 03:35:00.922633 15245 webserver.cc:460] Webserver started at http://127.27.40.188:44487/ using document root <none> and password file <none>
I20260809 03:35:00.922894 15245 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:00.922964 15245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:00.923112 15245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:00.924139 15245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/data/instance:
uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a"
format_stamp: "Formatted at 2026-08-09 03:35:00 on dist-test-slave-gkw7"
server_key: "dd1f24f7fed5a64a1e905ee560e663c4"
server_key_iv: "f2607c79342d50eea18e1cd1626d8677"
server_key_version: "encryptionkey@0"
I20260809 03:35:00.924494 15245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/wal/instance:
uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a"
format_stamp: "Formatted at 2026-08-09 03:35:00 on dist-test-slave-gkw7"
server_key: "dd1f24f7fed5a64a1e905ee560e663c4"
server_key_iv: "f2607c79342d50eea18e1cd1626d8677"
server_key_version: "encryptionkey@0"
I20260809 03:35:00.925844 15245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:35:00.926581 15267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:00.926760 15245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:35:00.926867 15245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/wal
uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a"
format_stamp: "Formatted at 2026-08-09 03:35:00 on dist-test-slave-gkw7"
server_key: "dd1f24f7fed5a64a1e905ee560e663c4"
server_key_iv: "f2607c79342d50eea18e1cd1626d8677"
server_key_version: "encryptionkey@0"
I20260809 03:35:00.926939 15245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:00.948426 15245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:00.950564 15245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:00.950738 15245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:00.954756 15245 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.188:46179
I20260809 03:35:00.954820 15347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.188:46179 every 8 connection(s)
I20260809 03:35:00.955233 15245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/data/info.pb
I20260809 03:35:00.956045 15136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } attempt: 2
I20260809 03:35:00.956135 15348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:00.958731 15348 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:35:00 dist-test-slave-gkw7 krb5kdc[15014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for kudu/127.27.40.188@KRBTEST.COM
Aug 09 03:35:00 dist-test-slave-gkw7 krb5kdc[15014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.188@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:00.965353 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 15245
I20260809 03:35:00.965456 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246339702443-27810-0/minicluster-data/master-2/wal/instance
I20260809 03:35:00.965655 27810 external_mini_cluster.cc:1428] Setting key f7350eddd4ff8c6034ba74cf4acc49ee
I20260809 03:35:00.966169 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.956388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:46179 (local address 127.0.0.1:55628)
0809 03:35:00.956477 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.956491 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:00.956553 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:00.957631 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:00.957632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:00.957784 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:00.957832 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.957834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.959404 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.959406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.960054 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.960056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.960687 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.960688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.960741 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.961214 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:00.961218 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:00.962638 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:00.964661 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:00.964661 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:00.964662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.964927 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.965193 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:00.965194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:00.965194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.965278 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.965783 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:00.965784 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:00.965832 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 03:35:00.965915 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 03:35:00.966028 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 03:35:00.966398 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.956560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55628 (local address 127.27.40.188:46179)
0809 03:35:00.956716 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.956726 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.956759 (+ 33us) server_negotiation.cc:407] Connection header received
0809 03:35:00.957363 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.957372 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.957436 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.957641 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:00.957936 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.959290 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.960146 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.960615 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.960699 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.962725 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:00.962733 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:00.962733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:00.962740 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:00.964570 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.965006 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.965008 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.965009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.965110 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.965365 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.965366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.965367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.965687 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:00.965787 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.966001 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.966140 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260809 03:35:00.967628 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.959425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52658 (local address 127.27.40.190:35165)
0809 03:35:00.959467 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.959469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.959658 (+ 189us) server_negotiation.cc:407] Connection header received
0809 03:35:00.959724 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.959725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.959735 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.959790 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:00.960258 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.960734 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.961692 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.961917 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.961971 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.964276 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:00.964278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:00.964278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:00.964284 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:00.966088 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.966722 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.966723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.966724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.966784 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.966976 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.966978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.966978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.967210 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:00.967290 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.967476 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.967553 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 03:35:00.967913 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.959392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:35165 (local address 127.0.0.1:52658)
0809 03:35:00.959535 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.959575 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:00.959664 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:00.959813 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:00.959816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:00.960073 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:00.960174 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.960175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.960833 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.960834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.961599 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.961602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.962005 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.962006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.962089 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.962597 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:00.962605 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:00.964196 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:00.966336 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:00.966343 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:00.966345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.966628 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.966837 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:00.966838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:00.966838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.966897 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.967281 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:00.967285 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:00.967334 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 03:35:00.967702 (+ 368us) client_negotiation.cc:240] Negotiation successful
0809 03:35:00.967819 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260809 03:35:00.968379 15136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9: Bootstrap starting.
I20260809 03:35:00.969328 15136 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:00.969734 15136 log.cc:824] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:00.970391 15348 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:35:00.970623 15136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9: No bootstrap required, opened a new log
Aug 09 03:35:00 dist-test-slave-gkw7 krb5kdc[15014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.188@KRBTEST.COM
I20260809 03:35:00.972824 15136 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } }
I20260809 03:35:00.972970 15136 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:00.973031 15136 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb2e9405c2ca4c62bd4b8102de03d2b9, State: Initialized, Role: FOLLOWER
I20260809 03:35:00.973160 15136 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } }
I20260809 03:35:00.973666 15136 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:00.973798 15355 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } } }
I20260809 03:35:00.973930 15355 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9 [sys.catalog]: This master's current role is: FOLLOWER
Aug 09 03:35:00 dist-test-slave-gkw7 krb5kdc[15014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.188@KRBTEST.COM for kudu/127.27.40.189@KRBTEST.COM
W20260809 03:35:00.975670 15377 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 03:35:00.975709 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.966171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55632 (local address 127.27.40.188:46179)
0809 03:35:00.966483 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.966485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.966492 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:35:00.966509 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.966510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.966519 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.966570 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:00.967361 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.967911 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.968905 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.969109 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.969181 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.971871 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:00.971873 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:00.971873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:00.971880 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:00.973436 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.974546 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.974547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.974548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.974630 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.974906 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.974907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.974908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.975062 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:00.975225 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.975520 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.975601 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":93,"threads_started":1}
W20260809 03:35:00.975952 15377 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:35:00.978293 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.970660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:43649 (local address 127.0.0.1:40194)
0809 03:35:00.970881 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.970891 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:00.970955 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:00.971085 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:00.971086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:00.971223 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:00.971248 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.971249 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.971943 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.971944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.972692 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.972694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.973214 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.973216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.973284 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.973707 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:00.973712 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:00.975157 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:00.977069 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:00.977070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:00.977070 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.977277 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.977566 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:00.977566 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:00.977567 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.977644 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.977990 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:00.977991 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:00.978036 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 03:35:00.978114 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 03:35:00.978206 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260809 03:35:00.978336 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.970774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40194 (local address 127.27.40.189:43649)
0809 03:35:00.970817 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.970819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.970933 (+ 114us) server_negotiation.cc:407] Connection header received
0809 03:35:00.971022 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.971023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.971034 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.971084 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:00.971328 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.971858 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.972818 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.973132 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.973210 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.975263 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:00.975264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:00.975264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:00.975272 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:00.976969 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.977370 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.977372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.977372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.977455 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.977743 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.977744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.977745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.977907 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:00.977984 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.978159 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.978245 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 03:35:00.978880 15348 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:35:00.976552 15037 master_runner.cc:186] Error getting master registration for 127.27.40.189:43649: OK, instance_id { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" instance_seqno: 1786246500713474 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 03:35:00.979490 15037 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 09 03:35:00 dist-test-slave-gkw7 krb5kdc[15014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.188@KRBTEST.COM for kudu/127.27.40.188@KRBTEST.COM
Aug 09 03:35:00 dist-test-slave-gkw7 krb5kdc[15014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:00.986793 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.979120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:46179 (local address 127.0.0.1:55636)
0809 03:35:00.979158 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.979167 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:00.979201 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:00.979703 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:00.979704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:00.979851 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:00.979885 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.979887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.980733 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.980734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.981207 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.981209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.981587 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.981588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.981637 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.982052 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:00.982057 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:00.983470 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:00.985607 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:00.985608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:00.985609 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.985830 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.986031 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:00.986032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:00.986032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.986083 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.986407 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:00.986408 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:00.986458 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 03:35:00.986544 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 03:35:00.986666 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 03:35:00.986881 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.979172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55636 (local address 127.27.40.188:46179)
0809 03:35:00.979571 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.979573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.979581 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:35:00.979600 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.979601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.979610 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.979668 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:00.979990 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.980647 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.981290 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.981483 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.981579 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.983563 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:00.983564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:00.983565 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:00.983573 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:00.985331 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.985925 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.985926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.985926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.985980 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.986159 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.986160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.986161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.986321 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:00.986392 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.986588 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.986726 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":378}
I20260809 03:35:00.988407 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.976670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52674 (local address 127.27.40.190:35165)
0809 03:35:00.976716 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.976718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.979764 (+ 3046us) server_negotiation.cc:407] Connection header received
0809 03:35:00.979832 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.979833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.979843 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.979898 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:00.980479 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.981223 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.982267 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.982580 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.982670 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.985300 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:00.985301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:00.985302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:00.985310 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:00.987189 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.987568 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.987569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.987569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.987626 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.987857 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.987858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.987858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.987976 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:00.988169 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.988217 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.988296 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 03:35:00.988834 15348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a: Bootstrap starting.
I20260809 03:35:00.989849 15348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:00.990163 15348 log.cc:824] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:00.990993 15348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a: No bootstrap required, opened a new log
Aug 09 03:35:00 dist-test-slave-gkw7 krb5kdc[15014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.189@KRBTEST.COM
I20260809 03:35:00.992987 15348 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } }
I20260809 03:35:00.993152 15348 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:00.993187 15348 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bf19f68c4b4ef3ad9d082a8f736a5a, State: Initialized, Role: FOLLOWER
I20260809 03:35:00.993309 15348 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } }
I20260809 03:35:00.993678 15348 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:00.993656 15380 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } } }
I20260809 03:35:00.993753 15380 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a [sys.catalog]: This master's current role is: FOLLOWER
W20260809 03:35:00.995538 15395 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:35:00.996394 15395 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:35:00.997105 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.988997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40210 (local address 127.27.40.189:43649)
0809 03:35:00.989037 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.989039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.989079 (+ 40us) server_negotiation.cc:407] Connection header received
0809 03:35:00.989154 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.989155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.989164 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.989214 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:00.989728 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.990189 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.991017 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.991220 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.991295 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.993453 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:00.993455 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:00.993455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:00.993462 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:00.995349 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.995903 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.995904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.995905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.995965 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.996216 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.996217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.996217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.996345 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:00.996550 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.996938 (+ 388us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.997018 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:35:00.998672 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:35:00.998649 15245 master_runner.cc:186] Error getting master registration for 127.27.40.189:43649: OK, instance_id { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" instance_seqno: 1786246500713474 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 03:35:01.001096 15245 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 03:35:01.008836 15236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } attempt: 3
I20260809 03:35:01.012708 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:01.001850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52680 (local address 127.27.40.190:35165)
0809 03:35:01.001893 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:01.001894 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:01.002269 (+ 375us) server_negotiation.cc:407] Connection header received
0809 03:35:01.002375 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:01.002376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:01.002387 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:01.002417 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:01.003163 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:01.003911 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:01.004965 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:01.005256 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:01.005300 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:01.009330 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:01.009331 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:01.009332 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:01.009340 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:01.011120 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:01.011829 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:01.011831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:01.011831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:01.011886 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:01.012078 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:01.012079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:01.012079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:01.012208 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:01.012288 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:01.012512 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 03:35:01.012609 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 09 03:35:01 dist-test-slave-gkw7 krb5kdc[15014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.189@KRBTEST.COM for kudu/127.27.40.188@KRBTEST.COM
I20260809 03:35:01.016494 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:01.002100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40226 (local address 127.27.40.189:43649)
0809 03:35:01.002139 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:01.002141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:01.005702 (+ 3561us) server_negotiation.cc:407] Connection header received
0809 03:35:01.006315 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:01.006316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:01.006327 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:01.006357 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:01.006983 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:01.007709 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:01.008726 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:01.009019 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:01.009058 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:01.013218 (+ 4160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:01.013219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:01.013220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:01.013227 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:01.015131 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:01.015583 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:01.015584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:01.015585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:01.015640 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:01.015865 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:01.015866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:01.015866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:01.015998 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:01.016069 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:01.016292 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 03:35:01.016378 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 03:35:01.018356 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:01.002275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55652 (local address 127.27.40.188:46179)
0809 03:35:01.002316 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:01.002317 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:01.005702 (+ 3385us) server_negotiation.cc:407] Connection header received
0809 03:35:01.005803 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:01.005804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:01.005814 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:01.005843 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:01.007238 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:01.008013 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:01.009935 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:01.010210 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:01.010251 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:01.011684 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:01.011685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:01.011686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:01.011692 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:01.013425 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:01.016329 (+ 2904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:01.016330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:01.016331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:01.016389 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:01.017684 (+ 1295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:01.017685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:01.017685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:01.017834 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:01.017906 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:01.018136 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 03:35:01.018241 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 03:35:01.020282 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:01.009109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:46179 (local address 127.0.0.1:55660)
0809 03:35:01.009154 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:01.009165 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:01.009216 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:01.009491 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:01.009492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:01.009726 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:01.009766 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:01.009768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:01.010746 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:01.010747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:01.011589 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:01.011592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:01.012105 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:01.012106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:01.012162 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:01.013672 (+ 1510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:01.013677 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:01.016711 (+ 3034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:01.019175 (+ 2464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:01.019176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:01.019176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:01.019392 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:01.019628 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:01.019629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:01.019629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:01.019688 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:01.019959 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:01.019960 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:01.020002 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 03:35:01.020088 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 03:35:01.020187 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 03:35:01.020296 15353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:01.009249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55660 (local address 127.27.40.188:46179)
0809 03:35:01.009308 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:01.009310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:01.009319 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:35:01.009340 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:01.009341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:01.009362 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:01.009431 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:01.009854 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:01.010649 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:01.011673 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:01.011988 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:01.012049 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:01.017248 (+ 5199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:01.017249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:01.017249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:01.017257 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:01.019088 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:01.019489 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:01.019490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:01.019490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:01.019555 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:01.019768 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:01.019769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:01.019769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:01.019883 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:01.019954 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:01.020114 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 03:35:01.020187 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 03:35:01.021821 15236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc: Bootstrap starting.
I20260809 03:35:01.022584 15236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:01.022887 15236 log.cc:824] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:01.023677 15236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc: No bootstrap required, opened a new log
I20260809 03:35:01.025882 15236 raft_consensus.cc:348] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } }
I20260809 03:35:01.026013 15236 raft_consensus.cc:374] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:01.026096 15236 raft_consensus.cc:729] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f93b35768efa4e31a5d0c12d5f9178cc, State: Initialized, Role: FOLLOWER
I20260809 03:35:01.026301 15236 consensus_queue.cc:260] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } }
I20260809 03:35:01.026587 15412 sys_catalog.cc:455] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } } }
I20260809 03:35:01.026648 15236 sys_catalog.cc:564] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:01.026729 15412 sys_catalog.cc:458] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [sys.catalog]: This master's current role is: FOLLOWER
W20260809 03:35:01.029152 15428 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:35:01.029258 15428 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:35:01.030582 15143 master_runner.cc:263] No leader master found from master f93b35768efa4e31a5d0c12d5f9178cc
I20260809 03:35:01.030632 15143 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 03:35:01.117101 15412 raft_consensus.cc:482] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:01.117240 15412 raft_consensus.cc:504] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } }
I20260809 03:35:01.117512 15412 leader_election.cc:288] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb2e9405c2ca4c62bd4b8102de03d2b9 (127.27.40.190:35165), d4bf19f68c4b4ef3ad9d082a8f736a5a (127.27.40.188:46179)
I20260809 03:35:01.118046 15098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" is_pre_election: true
I20260809 03:35:01.118185 15307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" is_pre_election: true
I20260809 03:35:01.118263 15098 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f93b35768efa4e31a5d0c12d5f9178cc in term 0.
I20260809 03:35:01.118338 15307 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f93b35768efa4e31a5d0c12d5f9178cc in term 0.
I20260809 03:35:01.118528 15174 leader_election.cc:302] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4bf19f68c4b4ef3ad9d082a8f736a5a, f93b35768efa4e31a5d0c12d5f9178cc; no voters:
I20260809 03:35:01.118654 15412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:01.118713 15412 raft_consensus.cc:482] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:01.118757 15412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:01.119735 15412 raft_consensus.cc:504] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } }
I20260809 03:35:01.119884 15412 leader_election.cc:288] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [CANDIDATE]: Term 1 election: Requested vote from peers bb2e9405c2ca4c62bd4b8102de03d2b9 (127.27.40.190:35165), d4bf19f68c4b4ef3ad9d082a8f736a5a (127.27.40.188:46179)
I20260809 03:35:01.120054 15098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9"
I20260809 03:35:01.120064 15307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a"
I20260809 03:35:01.120131 15307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:01.120131 15098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:01.121038 15307 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f93b35768efa4e31a5d0c12d5f9178cc in term 1.
I20260809 03:35:01.121038 15098 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f93b35768efa4e31a5d0c12d5f9178cc in term 1.
I20260809 03:35:01.121214 15174 leader_election.cc:302] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4bf19f68c4b4ef3ad9d082a8f736a5a, f93b35768efa4e31a5d0c12d5f9178cc; no voters:
I20260809 03:35:01.121291 15412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:01.121455 15412 raft_consensus.cc:686] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [term 1 LEADER]: Becoming Leader. State: Replica: f93b35768efa4e31a5d0c12d5f9178cc, State: Running, Role: LEADER
I20260809 03:35:01.121606 15412 consensus_queue.cc:237] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } }
I20260809 03:35:01.123241 15431 sys_catalog.cc:455] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader f93b35768efa4e31a5d0c12d5f9178cc. Latest consensus state: current_term: 1 leader_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } } }
I20260809 03:35:01.123359 15431 sys_catalog.cc:458] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:01.123610 15433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:01.123871 15433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:01.125844 15307 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a [term 1 FOLLOWER]: Refusing update from remote peer f93b35768efa4e31a5d0c12d5f9178cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:01.125862 15098 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9 [term 1 FOLLOWER]: Refusing update from remote peer f93b35768efa4e31a5d0c12d5f9178cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:01.126153 15412 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:01.126259 15431 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:01.127569 15380 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f93b35768efa4e31a5d0c12d5f9178cc. Latest consensus state: current_term: 1 leader_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } } }
I20260809 03:35:01.127657 15380 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:35:01.127939 15355 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f93b35768efa4e31a5d0c12d5f9178cc. Latest consensus state: current_term: 1 leader_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } } }
I20260809 03:35:01.128016 15355 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:35:01.128563 15412 sys_catalog.cc:455] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } } }
I20260809 03:35:01.128643 15412 sys_catalog.cc:458] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:01.129069 15433 catalog_manager.cc:1347] Generated new cluster ID: 71a742d312b54fb8b4877e9a91a5ca36
I20260809 03:35:01.129119 15433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:01.129084 15355 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } } }
I20260809 03:35:01.129155 15355 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:35:01.129269 15434 mvcc.cc:205] Tried to move back new op lower bound from 7316465668608716800 to 7316465668594884608. Current Snapshot: MvccSnapshot[applied={T|T < 7316465668608716800 or (T in {7316465668608716800})}]
I20260809 03:35:01.134858 15437 mvcc.cc:205] Tried to move back new op lower bound from 7316465668608716800 to 7316465668594884608. Current Snapshot: MvccSnapshot[applied={T|T < 7316465668608716800}]
I20260809 03:35:01.141891 15431 sys_catalog.cc:455] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } } }
I20260809 03:35:01.142004 15431 sys_catalog.cc:458] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:01.142402 15380 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2e9405c2ca4c62bd4b8102de03d2b9" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35165 } } peers { permanent_uuid: "f93b35768efa4e31a5d0c12d5f9178cc" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 43649 } } peers { permanent_uuid: "d4bf19f68c4b4ef3ad9d082a8f736a5a" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 46179 } } }
I20260809 03:35:01.142490 15380 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:35:01.142547 15436 mvcc.cc:205] Tried to move back new op lower bound from 7316465668608716800 to 7316465668594884608. Current Snapshot: MvccSnapshot[applied={T|T < 7316465668608716800}]
I20260809 03:35:01.185806 15433 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:01.186582 15433 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:01.202663 15433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f93b35768efa4e31a5d0c12d5f9178cc: Generated new TSK 0
I20260809 03:35:01.202900 15433 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:35:01.976661 15377 catalog_manager.cc:1259] Loaded cluster ID: 71a742d312b54fb8b4877e9a91a5ca36
I20260809 03:35:01.976781 15377 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9: loading cluster ID for follower catalog manager: success
I20260809 03:35:01.977701 15377 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9: acquiring CA information for follower catalog manager: success
I20260809 03:35:01.978036 15377 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bb2e9405c2ca4c62bd4b8102de03d2b9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:35:01.996896 15395 catalog_manager.cc:1259] Loaded cluster ID: 71a742d312b54fb8b4877e9a91a5ca36
I20260809 03:35:01.996979 15395 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a: loading cluster ID for follower catalog manager: success
I20260809 03:35:01.997951 15395 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a: acquiring CA information for follower catalog manager: success
I20260809 03:35:01.998314 15395 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d4bf19f68c4b4ef3ad9d082a8f736a5a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:35:02.253629 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 15037
I20260809 03:35:02.277298 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 15143
I20260809 03:35:02.289506 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 15245
2026-08-09T03:35:02Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3571 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:35:02 dist-test-slave-gkw7 krb5kdc[15454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:35:02 dist-test-slave-gkw7 krb5kdc[15454](info): set up 2 sockets
Aug 09 03:35:02 dist-test-slave-gkw7 krb5kdc[15454](info): commencing operation
krb5kdc: starting...
W20260809 03:35:03.622997 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:35:03 dist-test-slave-gkw7 krb5kdc[15454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:35:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:35:03.746270 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:46831
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:36267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:46831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:03.828114 15472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:03.828318 15472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:03.828341 15472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:03.829392 15472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:03.829427 15472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:03.829440 15472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:03.829452 15472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:03.829464 15472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:03.830956 15472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:36267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:46831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:46831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:03.831152 15472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:03.831313 15472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:03.836170 15482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:03.836220 15480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:03.836166 15485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:03.836484 15472 server_base.cc:1034] running on GCE node
I20260809 03:35:03.836750 15472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:03.837054 15472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:03.838204 15472 hybrid_clock.cc:648] HybridClock initialized: now 1786246503838184 us; error 38 us; skew 500 ppm
Aug 09 03:35:03 dist-test-slave-gkw7 krb5kdc[15454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246503, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:03.840749 15472 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:35:03.841188 15472 webserver.cc:460] Webserver started at http://127.27.40.190:44487/ using document root <none> and password file <none>
I20260809 03:35:03.841436 15472 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:03.841507 15472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:03.841696 15472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:03.842767 15472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "f5e642bdf35d4be08a05e34cdbfbc2b0"
format_stamp: "Formatted at 2026-08-09 03:35:03 on dist-test-slave-gkw7"
server_key: "52f218ee3ef24cde30bc41f5994d6bd5"
server_key_iv: "689bde3a1f7014edf51b3118263ba7de"
server_key_version: "encryptionkey@0"
I20260809 03:35:03.843137 15472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "f5e642bdf35d4be08a05e34cdbfbc2b0"
format_stamp: "Formatted at 2026-08-09 03:35:03 on dist-test-slave-gkw7"
server_key: "52f218ee3ef24cde30bc41f5994d6bd5"
server_key_iv: "689bde3a1f7014edf51b3118263ba7de"
server_key_version: "encryptionkey@0"
I20260809 03:35:03.844488 15472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:35:03.845203 15493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:03.845386 15472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:35:03.845485 15472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "f5e642bdf35d4be08a05e34cdbfbc2b0"
format_stamp: "Formatted at 2026-08-09 03:35:03 on dist-test-slave-gkw7"
server_key: "52f218ee3ef24cde30bc41f5994d6bd5"
server_key_iv: "689bde3a1f7014edf51b3118263ba7de"
server_key_version: "encryptionkey@0"
I20260809 03:35:03.845593 15472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:03.865590 15472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:03.867535 15472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:03.867709 15472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:03.871984 15472 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:46831
I20260809 03:35:03.872030 15559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:46831 every 8 connection(s)
I20260809 03:35:03.872419 15472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:03.873378 15560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:03.875923 15560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0: Bootstrap starting.
I20260809 03:35:03.876514 15560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:03.876775 15560 log.cc:824] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:03.877526 15560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0: No bootstrap required, opened a new log
I20260809 03:35:03.879201 15560 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e642bdf35d4be08a05e34cdbfbc2b0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 46831 } }
I20260809 03:35:03.879367 15560 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:03.879392 15560 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5e642bdf35d4be08a05e34cdbfbc2b0, State: Initialized, Role: FOLLOWER
I20260809 03:35:03.879565 15560 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e642bdf35d4be08a05e34cdbfbc2b0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 46831 } }
I20260809 03:35:03.879660 15560 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:03.879694 15560 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:03.879746 15560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:03.880291 15560 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e642bdf35d4be08a05e34cdbfbc2b0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 46831 } }
I20260809 03:35:03.880383 15560 leader_election.cc:302] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5e642bdf35d4be08a05e34cdbfbc2b0; no voters:
I20260809 03:35:03.880528 15560 leader_election.cc:288] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:03.880589 15564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:03.880792 15564 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [term 1 LEADER]: Becoming Leader. State: Replica: f5e642bdf35d4be08a05e34cdbfbc2b0, State: Running, Role: LEADER
I20260809 03:35:03.880818 15560 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:03.880887 15564 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e642bdf35d4be08a05e34cdbfbc2b0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 46831 } }
I20260809 03:35:03.881232 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 15472
I20260809 03:35:03.881348 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:35:03.881544 15565 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5e642bdf35d4be08a05e34cdbfbc2b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e642bdf35d4be08a05e34cdbfbc2b0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 46831 } } }
I20260809 03:35:03.881593 27810 external_mini_cluster.cc:1428] Setting key 78d832c414d866f41a966bdfb36741ff
I20260809 03:35:03.881722 15565 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:03.881956 15566 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5e642bdf35d4be08a05e34cdbfbc2b0. Latest consensus state: current_term: 1 leader_uuid: "f5e642bdf35d4be08a05e34cdbfbc2b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e642bdf35d4be08a05e34cdbfbc2b0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 46831 } } }
I20260809 03:35:03.882042 15573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:03.882741 15573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:03.883409 15566 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:03.884554 15573 catalog_manager.cc:1347] Generated new cluster ID: 286386668e3d48a485044af026347468
I20260809 03:35:03.884617 15573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 03:35:03 dist-test-slave-gkw7 krb5kdc[15454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:03.892741 15581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.882235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38876 (local address 127.27.40.190:46831)
0809 03:35:03.882793 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:03.882803 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:03.882837 (+ 34us) server_negotiation.cc:407] Connection header received
0809 03:35:03.882959 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:03.882966 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:03.883025 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:03.883183 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:03.883897 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.885028 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.885901 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.886334 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.886463 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.888826 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:03.888830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:03.888831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:03.888850 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:03.891079 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:03.891479 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:03.891482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:03.891483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:03.891591 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:03.891820 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:03.891821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:03.891822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:03.892067 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:03.892240 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:03.892320 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 03:35:03.892566 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":73,"threads_started":1}
I20260809 03:35:03.893750 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:35:03.903528 15573 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:03.904299 15573 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:03.910140 15573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5e642bdf35d4be08a05e34cdbfbc2b0: Generated new TSK 0
I20260809 03:35:03.910360 15573 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:35:03.947911 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 15472
2026-08-09T03:35:03Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1655 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:35:04 dist-test-slave-gkw7 krb5kdc[15593](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:35:04 dist-test-slave-gkw7 krb5kdc[15593](info): set up 2 sockets
Aug 09 03:35:04 dist-test-slave-gkw7 krb5kdc[15593](info): commencing operation
krb5kdc: starting...
W20260809 03:35:05.173153 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:35:05 dist-test-slave-gkw7 krb5kdc[15593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:35:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:35:05.298000 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:40823
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:40823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:05.380425 15610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:05.380622 15610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:05.380645 15610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:05.381856 15610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:05.381899 15610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:05.381914 15610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:05.381927 15610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:05.381940 15610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:05.383479 15610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:40823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:40823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:05.383723 15610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:05.383955 15610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:05.388885 15617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:05.388839 15616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:05.389035 15610 server_base.cc:1034] running on GCE node
W20260809 03:35:05.388823 15619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:05.389345 15610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:05.389744 15610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:05.390913 15610 hybrid_clock.cc:648] HybridClock initialized: now 1786246505390909 us; error 35 us; skew 500 ppm
Aug 09 03:35:05 dist-test-slave-gkw7 krb5kdc[15593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:05.393397 15610 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:35:05.393936 15610 webserver.cc:460] Webserver started at http://127.27.40.190:40167/ using document root <none> and password file <none>
I20260809 03:35:05.394194 15610 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:05.394274 15610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:05.394423 15610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:05.395709 15610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "90af873d5eba42bfb25d219be3957225"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "c3a9e75c9e3ae0a7b11f6eae6e92ab39"
server_key_iv: "b4ea70fdb13a8dc69ecd3cfb99b3b429"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.396155 15610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "90af873d5eba42bfb25d219be3957225"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "c3a9e75c9e3ae0a7b11f6eae6e92ab39"
server_key_iv: "b4ea70fdb13a8dc69ecd3cfb99b3b429"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.397524 15610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:35:05.398336 15628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.398496 15610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:35:05.398613 15610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "90af873d5eba42bfb25d219be3957225"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "c3a9e75c9e3ae0a7b11f6eae6e92ab39"
server_key_iv: "b4ea70fdb13a8dc69ecd3cfb99b3b429"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.398696 15610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:05.410768 15610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:05.412439 15610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:05.412639 15610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:05.416975 15610 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:40823
I20260809 03:35:05.417029 15701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:40823 every 8 connection(s)
I20260809 03:35:05.417343 15610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:05.418362 15702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:05.420548 15702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: Bootstrap starting.
I20260809 03:35:05.421144 15702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:05.421440 15702 log.cc:824] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:05.422144 15702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: No bootstrap required, opened a new log
I20260809 03:35:05.422739 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 15610
I20260809 03:35:05.422829 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:35:05.423035 27810 external_mini_cluster.cc:1428] Setting key e983cd76b410ca8d9b35448444b88113
I20260809 03:35:05.424335 15702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } }
I20260809 03:35:05.424505 15702 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:05.424546 15702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90af873d5eba42bfb25d219be3957225, State: Initialized, Role: FOLLOWER
I20260809 03:35:05.424743 15702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } }
I20260809 03:35:05.424831 15702 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:05.424853 15702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:05.424916 15702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:05.425992 15702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } }
I20260809 03:35:05.426098 15702 leader_election.cc:302] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90af873d5eba42bfb25d219be3957225; no voters:
I20260809 03:35:05.426249 15702 leader_election.cc:288] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:05.426296 15712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:05.426461 15712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 1 LEADER]: Becoming Leader. State: Replica: 90af873d5eba42bfb25d219be3957225, State: Running, Role: LEADER
I20260809 03:35:05.426486 15702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:05.426627 15712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } }
I20260809 03:35:05.426980 15713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90af873d5eba42bfb25d219be3957225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } } }
I20260809 03:35:05.426993 15714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90af873d5eba42bfb25d219be3957225. Latest consensus state: current_term: 1 leader_uuid: "90af873d5eba42bfb25d219be3957225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } } }
I20260809 03:35:05.427150 15713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:05.427163 15714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:05.427951 15722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:05.428447 15722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:05.430078 15722 catalog_manager.cc:1347] Generated new cluster ID: b177eedca0694c48bab4c6600f5ba101
I20260809 03:35:05.430132 15722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 03:35:05 dist-test-slave-gkw7 krb5kdc[15593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:05.435125 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.423554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57738 (local address 127.27.40.190:40823)
0809 03:35:05.423764 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.423776 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.423811 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:35:05.424297 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.424306 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.424369 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.424518 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:05.425305 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.426750 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.427882 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.428467 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.428584 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.431450 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:05.431454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:05.431454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:05.431465 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:05.433389 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.433822 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.433825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.433825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.433948 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.434184 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.434185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.434185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.434476 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:05.434624 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.434800 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.434935 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260809 03:35:05.436347 15722 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:05.437080 15722 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:05.443742 15722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: Generated new TSK 0
I20260809 03:35:05.443899 15722 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:35:05.474050 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40823
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:05.555737 15735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:05.555959 15735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:05.556000 15735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:05.557157 15735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:05.557224 15735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:05.557268 15735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:35:05.558763 15735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:05.559019 15735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:05.559247 15735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:05.564260 15750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:05.564332 15735 server_base.cc:1034] running on GCE node
W20260809 03:35:05.564266 15745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:05.564266 15743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:05.564668 15735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:05.564867 15735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:05.565999 15735 hybrid_clock.cc:648] HybridClock initialized: now 1786246505565982 us; error 33 us; skew 500 ppm
Aug 09 03:35:05 dist-test-slave-gkw7 krb5kdc[15593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:05.568292 15735 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:35:05.568783 15735 webserver.cc:460] Webserver started at http://127.27.40.129:40225/ using document root <none> and password file <none>
I20260809 03:35:05.569021 15735 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:05.569088 15735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:05.569259 15735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:05.570410 15735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "938ccc8f4b284b849441e438de98f599"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "79a59517dc60d678eaf4501d7fe2952b"
server_key_iv: "081b358f7fc2c4556d13a68d811c6b8c"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.570751 15735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "938ccc8f4b284b849441e438de98f599"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "79a59517dc60d678eaf4501d7fe2952b"
server_key_iv: "081b358f7fc2c4556d13a68d811c6b8c"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.572063 15735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 03:35:05.572796 15758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.572978 15735 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:35:05.573060 15735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "938ccc8f4b284b849441e438de98f599"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "79a59517dc60d678eaf4501d7fe2952b"
server_key_iv: "081b358f7fc2c4556d13a68d811c6b8c"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.573112 15735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:05.586512 15735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:05.588251 15735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:05.588357 15735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:05.588572 15735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:05.588819 15735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:05.588863 15735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.588884 15735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:05.588896 15735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.594996 15735 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:36095
I20260809 03:35:05.595052 15918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:36095 every 8 connection(s)
I20260809 03:35:05.595340 15735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:05.598968 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 15735
I20260809 03:35:05.599051 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:35:05.599225 27810 external_mini_cluster.cc:1428] Setting key 538fbf3df64afc52c0de7a3755c8bf01
Aug 09 03:35:05 dist-test-slave-gkw7 krb5kdc[15593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:05.605201 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.596047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:48679 (local address 127.27.40.190:40823)
0809 03:35:05.596113 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.596115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.596280 (+ 165us) server_negotiation.cc:407] Connection header received
0809 03:35:05.596746 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.596747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.596758 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.596813 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:05.597772 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.598263 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.599337 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.599624 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.599689 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.601968 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:05.601969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:05.601970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:05.601981 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:05.603727 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.604139 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.604141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.604141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.604207 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.604436 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.604437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.604438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.604604 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:05.604681 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.604974 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.605076 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 03:35:05.605203 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.595996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:48679)
0809 03:35:05.596159 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:05.596199 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:05.596662 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:05.596887 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:05.596890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:05.597425 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:05.597697 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.597702 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.598340 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.598341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.599228 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.599230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.599715 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.599716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.599853 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.600372 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:05.600387 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:05.601868 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:05.603830 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.603837 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.603839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.604057 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.604274 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.604274 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.604275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.604370 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.604673 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:05.604678 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:05.604785 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 03:35:05.604930 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 03:35:05.605049 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260809 03:35:05.605769 15919 heartbeater.cc:347] Connected to a master server at 127.27.40.190:40823
I20260809 03:35:05.605861 15919 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:05.606060 15919 heartbeater.cc:511] Master 127.27.40.190:40823 requested a full tablet report, sending...
I20260809 03:35:05.606608 15651 ts_manager.cc:194] Registered new tserver with Master: 938ccc8f4b284b849441e438de98f599 (127.27.40.129:36095)
I20260809 03:35:05.607210 15651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:48679
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:35:05.628360 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40823
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:05.717449 15926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:05.717760 15926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:05.717806 15926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:05.719081 15926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:05.719149 15926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:05.719192 15926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:35:05.720670 15926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:05.720935 15926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:05.721184 15926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:05.726361 15939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:05.726370 15942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:05.726420 15937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:05.726912 15926 server_base.cc:1034] running on GCE node
I20260809 03:35:05.727162 15926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:05.727357 15926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:05.728535 15926 hybrid_clock.cc:648] HybridClock initialized: now 1786246505728522 us; error 31 us; skew 500 ppm
Aug 09 03:35:05 dist-test-slave-gkw7 krb5kdc[15593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:05.731069 15926 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:35:05.731491 15926 webserver.cc:460] Webserver started at http://127.27.40.130:39585/ using document root <none> and password file <none>
I20260809 03:35:05.731745 15926 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:05.731817 15926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:05.731963 15926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:05.732980 15926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "edfea1f6d780425b8a4813df48f6219d"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "abe2abaaeeaa7e0b353250e3f04d038d"
server_key_iv: "92fac19e97afc7d95525113807574716"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.733347 15926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "edfea1f6d780425b8a4813df48f6219d"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "abe2abaaeeaa7e0b353250e3f04d038d"
server_key_iv: "92fac19e97afc7d95525113807574716"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.734760 15926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:05.735546 15952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.735795 15926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:05.735903 15926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "edfea1f6d780425b8a4813df48f6219d"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "abe2abaaeeaa7e0b353250e3f04d038d"
server_key_iv: "92fac19e97afc7d95525113807574716"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.735976 15926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:05.745270 15926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:05.747813 15926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:05.747995 15926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:05.748245 15926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:05.748572 15926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:05.748632 15926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.748677 15926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:05.748723 15926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.755050 15926 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:36537
I20260809 03:35:05.755132 16102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:36537 every 8 connection(s)
I20260809 03:35:05.755450 15926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
Aug 09 03:35:05 dist-test-slave-gkw7 krb5kdc[15593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:05.763826 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 15926
I20260809 03:35:05.763957 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:35:05.764127 27810 external_mini_cluster.cc:1428] Setting key 81c88180c48054211f187ac9da6729a7
I20260809 03:35:05.765249 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.756223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:42301 (local address 127.27.40.190:40823)
0809 03:35:05.756400 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.756403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.756578 (+ 175us) server_negotiation.cc:407] Connection header received
0809 03:35:05.757124 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.757126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.757138 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.757196 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:05.758210 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.758740 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.759782 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.759989 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.760070 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.762267 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:05.762269 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:05.762269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:05.762277 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:05.763657 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.764108 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.764109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.764110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.764171 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.764537 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.764538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.764539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.764673 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:05.764741 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.765082 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.765154 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260809 03:35:05.765498 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.756168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:42301)
0809 03:35:05.756431 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:05.756486 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:05.757020 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:05.757263 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:05.757267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:05.757908 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:05.758099 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.758104 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.758841 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.758843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.759651 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.759653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.760078 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.760080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.760211 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.760717 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:05.760731 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:05.762165 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:05.763751 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.763759 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.763761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.764046 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.764291 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.764292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.764293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.764442 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.764738 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:05.764744 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:05.764914 (+ 170us) client_negotiation.cc:769] Sending connection context
0809 03:35:05.765051 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 03:35:05.765240 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":90,"threads_started":1}
I20260809 03:35:05.765988 16103 heartbeater.cc:347] Connected to a master server at 127.27.40.190:40823
I20260809 03:35:05.766101 16103 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:05.766285 16103 heartbeater.cc:511] Master 127.27.40.190:40823 requested a full tablet report, sending...
I20260809 03:35:05.766696 15651 ts_manager.cc:194] Registered new tserver with Master: edfea1f6d780425b8a4813df48f6219d (127.27.40.130:36537)
I20260809 03:35:05.767228 15651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:42301
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:35:05.791941 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40823
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:05.873651 16109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:05.873854 16109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:05.873878 16109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:05.874980 16109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:05.875013 16109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:05.875080 16109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:35:05.876639 16109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:40823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:05.876823 16109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:05.876997 16109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:05.882139 16121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:05.882076 16117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:05.882067 16118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:05.882187 16109 server_base.cc:1034] running on GCE node
I20260809 03:35:05.882565 16109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:05.882859 16109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:05.884002 16109 hybrid_clock.cc:648] HybridClock initialized: now 1786246505883991 us; error 36 us; skew 500 ppm
Aug 09 03:35:05 dist-test-slave-gkw7 krb5kdc[15593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:05.886708 16109 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:35:05.887185 16109 webserver.cc:460] Webserver started at http://127.27.40.131:38417/ using document root <none> and password file <none>
I20260809 03:35:05.887444 16109 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:05.887488 16109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:05.887655 16109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:05.888691 16109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "008e0dbe29434159a95bf1656b084d1c"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "d578c1ec171b8f3e84a5da624bfbb7e4"
server_key_iv: "1108c452f06efb22381b82d2391d099a"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.889065 16109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "008e0dbe29434159a95bf1656b084d1c"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "d578c1ec171b8f3e84a5da624bfbb7e4"
server_key_iv: "1108c452f06efb22381b82d2391d099a"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.891263 16109 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 03:35:05.892005 16140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.892185 16109 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.892282 16109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "008e0dbe29434159a95bf1656b084d1c"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "d578c1ec171b8f3e84a5da624bfbb7e4"
server_key_iv: "1108c452f06efb22381b82d2391d099a"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.892370 16109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:05.900803 16109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:05.903191 16109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:05.903347 16109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:05.903563 16109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:05.903899 16109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:05.903952 16109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.903992 16109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:05.904018 16109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.909955 16109 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:39745
I20260809 03:35:05.910004 16299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:39745 every 8 connection(s)
I20260809 03:35:05.910354 16109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
Aug 09 03:35:05 dist-test-slave-gkw7 krb5kdc[15593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:05.916862 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 16109
I20260809 03:35:05.916958 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:35:05.917122 27810 external_mini_cluster.cc:1428] Setting key ff52ebc63d31a514ae8ff04861d19dce
I20260809 03:35:05.919946 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.911138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:49861 (local address 127.27.40.190:40823)
0809 03:35:05.911208 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.911211 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.911420 (+ 209us) server_negotiation.cc:407] Connection header received
0809 03:35:05.911872 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.911873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.911884 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.911936 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:05.912823 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.913313 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.914357 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.914592 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.914674 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.917343 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:05.917345 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:05.917345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:05.917354 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:05.918861 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.919227 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.919228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.919229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.919278 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.919449 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.919449 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.919449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.919546 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:05.919594 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.919826 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.919883 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 03:35:05.920104 16304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.911105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:49861)
0809 03:35:05.911305 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:05.911339 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:05.911794 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:05.911987 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:05.911990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:05.912560 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:05.912750 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.912754 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.913391 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.913392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.914284 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.914288 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.914686 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.914687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.914854 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.915530 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:05.915541 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:05.917234 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:05.918931 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.918937 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.918939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.919161 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.919319 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.919319 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.919319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.919400 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.919602 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:05.919607 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:05.919718 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 03:35:05.919798 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 03:35:05.919912 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":116,"threads_started":1}
I20260809 03:35:05.920538 16300 heartbeater.cc:347] Connected to a master server at 127.27.40.190:40823
I20260809 03:35:05.920630 16300 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:05.920774 16300 heartbeater.cc:511] Master 127.27.40.190:40823 requested a full tablet report, sending...
I20260809 03:35:05.921098 15651 ts_manager.cc:194] Registered new tserver with Master: 008e0dbe29434159a95bf1656b084d1c (127.27.40.131:39745)
I20260809 03:35:05.921432 15651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:49861
I20260809 03:35:05.929123 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:35:05.936576 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.930552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57750 (local address 127.27.40.190:40823)
0809 03:35:05.930615 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.930618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.930743 (+ 125us) server_negotiation.cc:407] Connection header received
0809 03:35:05.930782 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.930783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.930792 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.930832 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:05.931497 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.931966 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.932685 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.932866 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.932916 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.934251 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:05.934253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:05.934253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:05.934259 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:05.935601 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.935919 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.935920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.935920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.935970 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.936139 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.936139 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.936140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.936273 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:05.936345 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.936430 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.936486 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 03:35:05.938356 15651 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:35:05.940024 15651 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:05.946523 16317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.942305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:36095 (local address 127.0.0.1:41766)
0809 03:35:05.942574 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:05.942584 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:05.942627 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:05.942917 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:05.942920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:05.942923 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:05.943134 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.943137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.944067 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.944068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.945278 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.945281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.946202 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.946203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.946294 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.946308 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 03:35:05.946382 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 03:35:05.946432 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":55,"threads_started":1}
I20260809 03:35:05.946552 16318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.942441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41766 (local address 127.27.40.129:36095)
0809 03:35:05.942753 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.942758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.942767 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:35:05.942798 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.942799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.942812 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.942965 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:05.943222 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.943958 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.945373 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.946110 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.946192 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.946262 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.946410 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.946469 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":103,"threads_started":1}
I20260809 03:35:05.946709 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.942486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:39745 (local address 127.0.0.1:34368)
0809 03:35:05.942639 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:05.942650 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:05.942706 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:05.943041 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:05.943042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:05.943045 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:05.943119 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.943121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.944538 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.944539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.945520 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.945522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.946475 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.946476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.946552 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.946557 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:35:05.946588 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 03:35:05.946639 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":55,"threads_started":1}
I20260809 03:35:05.946935 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.942673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34368 (local address 127.27.40.131:39745)
0809 03:35:05.942854 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.942860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.942870 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:35:05.942910 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.942912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.942930 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.943208 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:05.943284 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.944415 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.945630 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.946355 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.946776 (+ 421us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.946827 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.946848 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.946878 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":105,"threads_started":1}
I20260809 03:35:05.947646 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.942260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:36537 (local address 127.0.0.1:42466)
0809 03:35:05.942465 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:05.942508 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:05.942574 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:05.943175 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:05.943176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:05.943178 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:05.943222 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.943223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.945600 (+ 2377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.945601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.946425 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.946427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.947382 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.947383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.947446 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.947450 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:35:05.947502 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 03:35:05.947552 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":83,"threads_started":1}
I20260809 03:35:05.947793 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.942362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42466 (local address 127.27.40.130:36537)
0809 03:35:05.942905 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.942909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.942917 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:35:05.942956 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.942959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.942973 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.943049 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:05.943293 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.944406 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.946513 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.947310 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.947380 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.947430 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.947667 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.947713 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":88,"threads_started":1}
I20260809 03:35:05.947712 15822 tablet_service.cc:1467] Processing CreateTablet for tablet 669b4f6e618f4d8a899943c07c0af6a7 (DEFAULT_TABLE table=test-table [id=75cfd613c9994570a48254572dad27c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:05.948125 15822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 669b4f6e618f4d8a899943c07c0af6a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:05.948053 16213 tablet_service.cc:1467] Processing CreateTablet for tablet 669b4f6e618f4d8a899943c07c0af6a7 (DEFAULT_TABLE table=test-table [id=75cfd613c9994570a48254572dad27c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:05.948386 16213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 669b4f6e618f4d8a899943c07c0af6a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:05.950501 16023 tablet_service.cc:1467] Processing CreateTablet for tablet 669b4f6e618f4d8a899943c07c0af6a7 (DEFAULT_TABLE table=test-table [id=75cfd613c9994570a48254572dad27c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:05.950839 16023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 669b4f6e618f4d8a899943c07c0af6a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:05.951102 16324 tablet_bootstrap.cc:492] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: Bootstrap starting.
I20260809 03:35:05.951424 16323 tablet_bootstrap.cc:492] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c: Bootstrap starting.
I20260809 03:35:05.951799 16324 tablet_bootstrap.cc:654] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:05.952077 16323 tablet_bootstrap.cc:654] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:05.952396 16324 log.cc:824] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:05.952643 16323 log.cc:824] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:05.953217 16324 tablet_bootstrap.cc:492] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: No bootstrap required, opened a new log
I20260809 03:35:05.953313 16324 ts_tablet_manager.cc:1397] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:35:05.953491 16323 tablet_bootstrap.cc:492] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c: No bootstrap required, opened a new log
I20260809 03:35:05.953586 16323 ts_tablet_manager.cc:1397] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:35:05.954330 16329 tablet_bootstrap.cc:492] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d: Bootstrap starting.
I20260809 03:35:05.955072 16329 tablet_bootstrap.cc:654] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:05.955503 16323 raft_consensus.cc:348] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } } peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } }
I20260809 03:35:05.955502 16324 raft_consensus.cc:348] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } } peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } }
I20260809 03:35:05.955631 16329 log.cc:824] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:05.955647 16323 raft_consensus.cc:374] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:05.955653 16324 raft_consensus.cc:374] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:05.955689 16323 raft_consensus.cc:729] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008e0dbe29434159a95bf1656b084d1c, State: Initialized, Role: FOLLOWER
I20260809 03:35:05.955689 16324 raft_consensus.cc:729] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 938ccc8f4b284b849441e438de98f599, State: Initialized, Role: FOLLOWER
I20260809 03:35:05.955842 16324 consensus_queue.cc:260] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } } peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } }
I20260809 03:35:05.955844 16323 consensus_queue.cc:260] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } } peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } }
I20260809 03:35:05.956133 16324 ts_tablet_manager.cc:1428] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:05.956148 16323 ts_tablet_manager.cc:1428] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 03:35:05.956307 16300 heartbeater.cc:503] Master 127.27.40.190:40823 was elected leader, sending a full tablet report...
I20260809 03:35:05.956442 16329 tablet_bootstrap.cc:492] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d: No bootstrap required, opened a new log
I20260809 03:35:05.956524 16329 ts_tablet_manager.cc:1397] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:35:05.958256 16329 raft_consensus.cc:348] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } } peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } }
I20260809 03:35:05.958375 16329 raft_consensus.cc:374] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:05.958396 16329 raft_consensus.cc:729] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edfea1f6d780425b8a4813df48f6219d, State: Initialized, Role: FOLLOWER
I20260809 03:35:05.958469 16329 consensus_queue.cc:260] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } } peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } }
I20260809 03:35:05.958657 16329 ts_tablet_manager.cc:1428] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:35:05.958745 16103 heartbeater.cc:503] Master 127.27.40.190:40823 was elected leader, sending a full tablet report...
I20260809 03:35:05.958796 15919 heartbeater.cc:503] Master 127.27.40.190:40823 was elected leader, sending a full tablet report...
W20260809 03:35:06.006032 16104 tablet.cc:2369] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:35:06.015352 16334 raft_consensus.cc:482] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:06.015524 16334 raft_consensus.cc:504] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } } peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } }
I20260809 03:35:06.016031 16334 leader_election.cc:288] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edfea1f6d780425b8a4813df48f6219d (127.27.40.130:36537), 938ccc8f4b284b849441e438de98f599 (127.27.40.129:36095)
I20260809 03:35:06.020013 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.016326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:48499 (local address 127.27.40.130:36537)
0809 03:35:06.016374 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.016376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.016508 (+ 132us) server_negotiation.cc:407] Connection header received
0809 03:35:06.016537 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.016539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.016549 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.016602 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:06.016749 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.017504 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.019055 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.019647 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.019728 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.019737 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.019904 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.019944 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 03:35:06.020013 16318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.016317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:52965 (local address 127.27.40.129:36095)
0809 03:35:06.016425 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.016427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.016434 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:35:06.016462 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.016463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.016498 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.016559 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:06.016755 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.017504 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.019054 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.019654 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.019728 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.019737 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.019905 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.019944 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 03:35:06.020028 16304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.016202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:36095 (local address 127.27.40.131:52965)
0809 03:35:06.016285 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.016299 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.016351 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.016584 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.016585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.016587 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:06.016668 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.016670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.017610 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.017611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.018958 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.018962 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.019741 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.019742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.019822 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.019824 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.019882 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.019931 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":87}
I20260809 03:35:06.020038 16337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.016225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:36537 (local address 127.27.40.131:48499)
0809 03:35:06.016422 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.016433 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.016498 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.016602 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.016602 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.016603 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:06.016686 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.016687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.017610 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.017610 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.018958 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.018962 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.019737 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.019738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.019819 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.019823 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.019882 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.019930 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":79,"thread_start_us":136,"threads_started":1}
I20260809 03:35:06.020375 15852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "669b4f6e618f4d8a899943c07c0af6a7" candidate_uuid: "008e0dbe29434159a95bf1656b084d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "938ccc8f4b284b849441e438de98f599" is_pre_election: true
I20260809 03:35:06.020565 15852 raft_consensus.cc:2454] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008e0dbe29434159a95bf1656b084d1c in term 0.
I20260809 03:35:06.020555 16048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "669b4f6e618f4d8a899943c07c0af6a7" candidate_uuid: "008e0dbe29434159a95bf1656b084d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edfea1f6d780425b8a4813df48f6219d" is_pre_election: true
I20260809 03:35:06.020754 16048 raft_consensus.cc:2454] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008e0dbe29434159a95bf1656b084d1c in term 0.
I20260809 03:35:06.020833 16145 leader_election.cc:302] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008e0dbe29434159a95bf1656b084d1c, 938ccc8f4b284b849441e438de98f599; no voters:
I20260809 03:35:06.020959 16334 raft_consensus.cc:2781] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:06.021034 16334 raft_consensus.cc:482] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:06.021059 16334 raft_consensus.cc:3037] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:06.021819 16334 raft_consensus.cc:504] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } } peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } }
I20260809 03:35:06.021965 16334 leader_election.cc:288] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [CANDIDATE]: Term 1 election: Requested vote from peers edfea1f6d780425b8a4813df48f6219d (127.27.40.130:36537), 938ccc8f4b284b849441e438de98f599 (127.27.40.129:36095)
I20260809 03:35:06.022156 15852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "669b4f6e618f4d8a899943c07c0af6a7" candidate_uuid: "008e0dbe29434159a95bf1656b084d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "938ccc8f4b284b849441e438de98f599"
I20260809 03:35:06.022223 15852 raft_consensus.cc:3037] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:06.022934 15852 raft_consensus.cc:2454] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008e0dbe29434159a95bf1656b084d1c in term 1.
I20260809 03:35:06.023063 16048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "669b4f6e618f4d8a899943c07c0af6a7" candidate_uuid: "008e0dbe29434159a95bf1656b084d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edfea1f6d780425b8a4813df48f6219d"
I20260809 03:35:06.023123 16145 leader_election.cc:302] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008e0dbe29434159a95bf1656b084d1c, 938ccc8f4b284b849441e438de98f599; no voters:
I20260809 03:35:06.023159 16048 raft_consensus.cc:3037] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:06.023248 16334 raft_consensus.cc:2781] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:06.023445 16334 raft_consensus.cc:686] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [term 1 LEADER]: Becoming Leader. State: Replica: 008e0dbe29434159a95bf1656b084d1c, State: Running, Role: LEADER
I20260809 03:35:06.023577 16334 consensus_queue.cc:237] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } } peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } }
I20260809 03:35:06.024006 16048 raft_consensus.cc:2454] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008e0dbe29434159a95bf1656b084d1c in term 1.
I20260809 03:35:06.024546 15650 catalog_manager.cc:5495] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 008e0dbe29434159a95bf1656b084d1c (127.27.40.131). New cstate: current_term: 1 leader_uuid: "008e0dbe29434159a95bf1656b084d1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } health_report { overall_health: HEALTHY } } }
I20260809 03:35:06.039044 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.036501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34372 (local address 127.27.40.131:39745)
0809 03:35:06.036572 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.036574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.036596 (+ 22us) server_negotiation.cc:407] Connection header received
0809 03:35:06.036638 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.036639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.036656 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.036741 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:06.036870 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.037340 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.038167 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.038419 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.038496 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.038652 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:06.038793 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:06.038893 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.038914 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.038934 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 03:35:06.041016 16048 raft_consensus.cc:1261] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d [term 1 FOLLOWER]: Refusing update from remote peer 008e0dbe29434159a95bf1656b084d1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:06.041023 15852 raft_consensus.cc:1261] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Refusing update from remote peer 008e0dbe29434159a95bf1656b084d1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:06.041360 16334 consensus_queue.cc:1035] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:06.041563 16339 consensus_queue.cc:1035] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:06.044373 16343 mvcc.cc:205] Tried to move back new op lower bound from 7316465688741072896 to 7316465688673361920. Current Snapshot: MvccSnapshot[applied={T|T < 7316465688741072896}]
I20260809 03:35:06.044835 16344 mvcc.cc:205] Tried to move back new op lower bound from 7316465688741072896 to 7316465688673361920. Current Snapshot: MvccSnapshot[applied={T|T < 7316465688741072896}]
I20260809 03:35:06.046506 15650 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:06.046613 15650 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:06.047433 15650 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: Sending DeleteTablet for 3 replicas of tablet 669b4f6e618f4d8a899943c07c0af6a7
I20260809 03:35:06.047787 16023 tablet_service.cc:1514] Processing DeleteTablet for tablet 669b4f6e618f4d8a899943c07c0af6a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:06 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:42466
I20260809 03:35:06.047835 15822 tablet_service.cc:1514] Processing DeleteTablet for tablet 669b4f6e618f4d8a899943c07c0af6a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:06 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:41766
I20260809 03:35:06.047835 16213 tablet_service.cc:1514] Processing DeleteTablet for tablet 669b4f6e618f4d8a899943c07c0af6a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:06 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:34368
I20260809 03:35:06.048125 16357 tablet_replica.cc:330] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: stopping tablet replica
I20260809 03:35:06.048136 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 15735
I20260809 03:35:06.048234 16357 raft_consensus.cc:2229] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:06.048337 16357 raft_consensus.cc:2258] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:06.048480 16356 tablet_replica.cc:330] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d: stopping tablet replica
I20260809 03:35:06.048563 16356 raft_consensus.cc:2229] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:06.051898 16356 raft_consensus.cc:2258] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:06.053875 16356 ts_tablet_manager.cc:1905] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:06.056485 16356 ts_tablet_manager.cc:1918] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:06.056612 16356 log.cc:1196] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/669b4f6e618f4d8a899943c07c0af6a7
I20260809 03:35:06.056999 16356 ts_tablet_manager.cc:1939] T 669b4f6e618f4d8a899943c07c0af6a7 P edfea1f6d780425b8a4813df48f6219d: Deleting consensus metadata
I20260809 03:35:06.057598 16358 tablet_replica.cc:330] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c: stopping tablet replica
I20260809 03:35:06.057617 15629 catalog_manager.cc:4841] TS edfea1f6d780425b8a4813df48f6219d (127.27.40.130:36537): tablet 669b4f6e618f4d8a899943c07c0af6a7 (table test-table [id=75cfd613c9994570a48254572dad27c9]) successfully deleted
I20260809 03:35:06.057844 16358 raft_consensus.cc:2229] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:35:06.058005 16358 raft_consensus.cc:2258] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:06.058784 16358 ts_tablet_manager.cc:1905] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:06.061082 16358 ts_tablet_manager.cc:1918] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:06.061167 16358 log.cc:1196] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/669b4f6e618f4d8a899943c07c0af6a7
I20260809 03:35:06.061702 16358 ts_tablet_manager.cc:1939] T 669b4f6e618f4d8a899943c07c0af6a7 P 008e0dbe29434159a95bf1656b084d1c: Deleting consensus metadata
I20260809 03:35:06.062927 15629 catalog_manager.cc:4841] TS 008e0dbe29434159a95bf1656b084d1c (127.27.40.131:39745): tablet 669b4f6e618f4d8a899943c07c0af6a7 (table test-table [id=75cfd613c9994570a48254572dad27c9]) successfully deleted
W20260809 03:35:06.069602 15630 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.40.129:36095 (error 108)
I20260809 03:35:06.069844 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 15926
I20260809 03:35:06.089092 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.088885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:36095 (local address 127.0.0.1:41770)
0809 03:35:06.088973 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.089039 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:36095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 03:35:06.089354 15630 catalog_manager.cc:4584] TS 938ccc8f4b284b849441e438de98f599 (127.27.40.129:36095): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 669b4f6e618f4d8a899943c07c0af6a7: Network error: Client connection negotiation failed: client connection to 127.27.40.129:36095: connect: Connection refused (error 111)
I20260809 03:35:06.093187 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 16109
I20260809 03:35:06.103175 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 15610
I20260809 03:35:06.115100 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:40823
--webserver_interface=127.27.40.190
--webserver_port=40167
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:40823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:06.197283 16361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:06.197499 16361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:06.197520 16361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:06.198612 16361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:06.198650 16361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:06.198664 16361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:06.198678 16361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:06.198688 16361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:06.200109 16361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:40823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:40823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=40167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:06.200295 16361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:06.200459 16361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:06.209831 16370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:06.209852 16374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:06.210012 16361 server_base.cc:1034] running on GCE node
W20260809 03:35:06.210139 16371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:06.210425 16361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:06.210731 16361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:06.211872 16361 hybrid_clock.cc:648] HybridClock initialized: now 1786246506211857 us; error 38 us; skew 500 ppm
Aug 09 03:35:06 dist-test-slave-gkw7 krb5kdc[15593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246506, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:06.220781 16361 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:35:06.221299 16361 webserver.cc:460] Webserver started at http://127.27.40.190:40167/ using document root <none> and password file <none>
I20260809 03:35:06.221611 16361 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:06.221691 16361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:06.223157 16361 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:06.223918 16382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:06.224102 16361 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:06.224213 16361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "90af873d5eba42bfb25d219be3957225"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "c3a9e75c9e3ae0a7b11f6eae6e92ab39"
server_key_iv: "b4ea70fdb13a8dc69ecd3cfb99b3b429"
server_key_version: "encryptionkey@0"
I20260809 03:35:06.224633 16361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:06.246768 16361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:06.249276 16361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:06.249450 16361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:06.253964 16361 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:40823
I20260809 03:35:06.254020 16456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:40823 every 8 connection(s)
I20260809 03:35:06.254452 16361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:06.256142 16457 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:35:06.256764 16457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: Bootstrap starting.
I20260809 03:35:06.261123 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 16361
I20260809 03:35:06.261441 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:36095
--local_ip_for_outbound_sockets=127.27.40.129
--tserver_master_addrs=127.27.40.190:40823
--webserver_port=40225
--webserver_interface=127.27.40.129
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:35:06.265756 16457 log.cc:824] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:06.267652 16457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:35:06.267977 16457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: Bootstrap complete.
I20260809 03:35:06.270402 16457 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } }
I20260809 03:35:06.270622 16457 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90af873d5eba42bfb25d219be3957225, State: Initialized, Role: FOLLOWER
I20260809 03:35:06.270792 16457 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } }
I20260809 03:35:06.270881 16457 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:06.270941 16457 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:06.271021 16457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:35:06.272028 16457 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } }
I20260809 03:35:06.272188 16457 leader_election.cc:302] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90af873d5eba42bfb25d219be3957225; no voters:
I20260809 03:35:06.272368 16457 leader_election.cc:288] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 03:35:06.272418 16462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 03:35:06.272629 16462 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 2 LEADER]: Becoming Leader. State: Replica: 90af873d5eba42bfb25d219be3957225, State: Running, Role: LEADER
I20260809 03:35:06.272717 16457 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:06.272782 16462 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } }
I20260809 03:35:06.273036 16462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90af873d5eba42bfb25d219be3957225. Latest consensus state: current_term: 2 leader_uuid: "90af873d5eba42bfb25d219be3957225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } } }
I20260809 03:35:06.273121 16462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:06.273038 16464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "90af873d5eba42bfb25d219be3957225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } } }
I20260809 03:35:06.273296 16464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:06.273391 16470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:06.274303 16470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:06.274470 16470 catalog_manager.cc:1259] Loaded cluster ID: b177eedca0694c48bab4c6600f5ba101
I20260809 03:35:06.274675 16470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:06.274925 16470 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 03:35:06.279597 16470 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 03:35:06.282100 16470 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:06.282290 16470 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: Loaded TSK: 0
I20260809 03:35:06.282511 16470 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 03:35:06.346479 16460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:06.346761 16460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:06.346807 16460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:06.347944 16460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:06.348007 16460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:06.348052 16460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:35:06.349566 16460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:36095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=40225
--tserver_master_addrs=127.27.40.190:40823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:06.349841 16460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:06.350075 16460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:06.356074 16506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:06.356184 16508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:06.356062 16505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:06.356576 16460 server_base.cc:1034] running on GCE node
I20260809 03:35:06.356825 16460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:06.357069 16460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:06.358232 16460 hybrid_clock.cc:648] HybridClock initialized: now 1786246506358208 us; error 37 us; skew 500 ppm
Aug 09 03:35:06 dist-test-slave-gkw7 krb5kdc[15593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246506, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:06.360721 16460 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:35:06.361186 16460 webserver.cc:460] Webserver started at http://127.27.40.129:40225/ using document root <none> and password file <none>
I20260809 03:35:06.361451 16460 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:06.361555 16460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:06.363000 16460 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:06.363749 16521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:06.363924 16460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:06.364027 16460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "938ccc8f4b284b849441e438de98f599"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "79a59517dc60d678eaf4501d7fe2952b"
server_key_iv: "081b358f7fc2c4556d13a68d811c6b8c"
server_key_version: "encryptionkey@0"
I20260809 03:35:06.364369 16460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:06.382413 16460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:06.384662 16460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:06.384835 16460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:06.385069 16460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:06.385565 16529 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 03:35:06.387566 16460 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 03:35:06.387627 16460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 03:35:06.387686 16460 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 03:35:06.388337 16460 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 03:35:06.388392 16460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 03:35:06.388461 16529 tablet_bootstrap.cc:492] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: Bootstrap starting.
I20260809 03:35:06.395913 16460 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:36095
I20260809 03:35:06.396373 16460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:06.397428 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 16460
I20260809 03:35:06.397758 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:36537
--local_ip_for_outbound_sockets=127.27.40.130
--tserver_master_addrs=127.27.40.190:40823
--webserver_port=39585
--webserver_interface=127.27.40.130
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:35:06.399488 16529 log.cc:824] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:06.402805 16529 tablet_bootstrap.cc:492] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:35:06.403203 16529 tablet_bootstrap.cc:492] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: Bootstrap complete.
I20260809 03:35:06.404623 16529 ts_tablet_manager.cc:1397] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260809 03:35:06.406664 16529 raft_consensus.cc:348] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } } peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } }
I20260809 03:35:06.406970 16529 raft_consensus.cc:729] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 938ccc8f4b284b849441e438de98f599, State: Initialized, Role: FOLLOWER
I20260809 03:35:06.407155 16676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:36095 every 8 connection(s)
I20260809 03:35:06.407258 16529 consensus_queue.cc:260] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } } peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } }
Aug 09 03:35:06 dist-test-slave-gkw7 krb5kdc[15593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246506, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:06.407526 16529 ts_tablet_manager.cc:1428] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:06.412537 16679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.397103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:44001)
0809 03:35:06.397436 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.397483 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.398228 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.400278 (+ 2050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.400283 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.400693 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:06.400959 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.400966 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.402777 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.402778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.404236 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.404239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.405260 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.405261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.405439 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.406075 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:06.406087 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:06.408033 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:06.410553 (+ 2520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:06.410561 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:06.410564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:06.410894 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:06.411245 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:06.411247 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:06.411247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:06.411391 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:06.411906 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:06.411913 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:06.412060 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.412159 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.412333 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":143,"threads_started":1}
I20260809 03:35:06.412927 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.397211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:44001 (local address 127.27.40.190:40823)
0809 03:35:06.399799 (+ 2588us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.399808 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.399849 (+ 41us) server_negotiation.cc:407] Connection header received
0809 03:35:06.399978 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.399985 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.400040 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.400198 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:06.401116 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.402632 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.404346 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.405007 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.405119 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.408163 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:06.408167 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:06.408168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:06.408181 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:06.410449 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:06.410996 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:06.411000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:06.411001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:06.411146 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:06.411478 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:06.411479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:06.411480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:06.411796 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:06.411875 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.412610 (+ 735us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.412747 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2533,"thread_start_us":92,"threads_started":1}
I20260809 03:35:06.413576 16677 heartbeater.cc:347] Connected to a master server at 127.27.40.190:40823
I20260809 03:35:06.413681 16677 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:06.413861 16677 heartbeater.cc:511] Master 127.27.40.190:40823 requested a full tablet report, sending...
I20260809 03:35:06.414647 16408 ts_manager.cc:194] Registered new tserver with Master: 938ccc8f4b284b849441e438de98f599 (127.27.40.129:36095)
I20260809 03:35:06.415752 16408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:44001
I20260809 03:35:06.416704 16677 heartbeater.cc:503] Master 127.27.40.190:40823 was elected leader, sending a full tablet report...
Aug 09 03:35:06 dist-test-slave-gkw7 krb5kdc[15593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246506, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for kudu/127.27.40.129@KRBTEST.COM
I20260809 03:35:06.423702 16686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.415215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:36095 (local address 127.0.0.1:38180)
0809 03:35:06.415548 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.415582 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.415669 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.415936 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.415941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.416240 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:06.416296 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.416299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.417633 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.417634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.418207 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.418209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.418854 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.418855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.418909 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.419386 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:06.419390 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:06.420649 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:06.422661 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:06.422669 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:06.422671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:06.422869 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:06.423081 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:06.423082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:06.423082 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:06.423139 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:06.423433 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:06.423438 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:06.423479 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.423540 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.423636 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":102,"threads_started":1}
I20260809 03:35:06.423902 16688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.415284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38180 (local address 127.27.40.129:36095)
0809 03:35:06.415480 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.415484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.415671 (+ 187us) server_negotiation.cc:407] Connection header received
0809 03:35:06.415754 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.415756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.415771 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.415846 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:06.416376 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.417503 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.418283 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.418770 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.418852 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.420728 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:06.420736 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:06.420737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:06.420744 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:06.422541 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:06.422948 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:06.422951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:06.422951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:06.423015 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:06.423196 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:06.423197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:06.423197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:06.423369 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:06.423449 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.423585 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.423688 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":170,"thread_start_us":90,"threads_started":1}
I20260809 03:35:06.424208 16584 tablet_service.cc:1514] Processing DeleteTablet for tablet 669b4f6e618f4d8a899943c07c0af6a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:06 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:38180
I20260809 03:35:06.424672 16689 tablet_replica.cc:330] stopping tablet replica
I20260809 03:35:06.424803 16689 raft_consensus.cc:2229] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:06.424886 16689 raft_consensus.cc:2258] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:06.425201 16689 ts_tablet_manager.cc:1905] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:06.427326 16689 ts_tablet_manager.cc:1918] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:06.427438 16689 log.cc:1196] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal/wals/669b4f6e618f4d8a899943c07c0af6a7
I20260809 03:35:06.427688 16689 ts_tablet_manager.cc:1939] T 669b4f6e618f4d8a899943c07c0af6a7 P 938ccc8f4b284b849441e438de98f599: Deleting consensus metadata
I20260809 03:35:06.428185 16386 catalog_manager.cc:4841] TS 938ccc8f4b284b849441e438de98f599 (127.27.40.129:36095): tablet 669b4f6e618f4d8a899943c07c0af6a7 (table test-table [id=75cfd613c9994570a48254572dad27c9]) successfully deleted
W20260809 03:35:06.489740 16681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:06.489936 16681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:06.489960 16681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:06.491429 16681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:06.491487 16681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:06.491516 16681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:35:06.493043 16681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:36537
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=39585
--tserver_master_addrs=127.27.40.190:40823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:06.493291 16681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:06.493525 16681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:06.498646 16695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:06.498776 16681 server_base.cc:1034] running on GCE node
W20260809 03:35:06.498649 16694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:06.498665 16697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:06.499075 16681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:06.499281 16681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:06.500414 16681 hybrid_clock.cc:648] HybridClock initialized: now 1786246506500396 us; error 39 us; skew 500 ppm
Aug 09 03:35:06 dist-test-slave-gkw7 krb5kdc[15593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246506, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:06.502962 16681 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:35:06.503369 16681 webserver.cc:460] Webserver started at http://127.27.40.130:39585/ using document root <none> and password file <none>
I20260809 03:35:06.503620 16681 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:06.503677 16681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:06.505024 16681 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:06.505674 16705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:06.505879 16681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 03:35:06.505975 16681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "edfea1f6d780425b8a4813df48f6219d"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "abe2abaaeeaa7e0b353250e3f04d038d"
server_key_iv: "92fac19e97afc7d95525113807574716"
server_key_version: "encryptionkey@0"
I20260809 03:35:06.506330 16681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:06.526533 16681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:06.528658 16681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:06.528774 16681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:06.528995 16681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:06.529280 16681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:06.529313 16681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:06.529335 16681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:06.529347 16681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:06.535534 16681 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:36537
I20260809 03:35:06.535586 16857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:36537 every 8 connection(s)
I20260809 03:35:06.535909 16681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
Aug 09 03:35:06 dist-test-slave-gkw7 krb5kdc[15593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246506, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:06.544413 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 16681
I20260809 03:35:06.544683 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:39745
--local_ip_for_outbound_sockets=127.27.40.131
--tserver_master_addrs=127.27.40.190:40823
--webserver_port=38417
--webserver_interface=127.27.40.131
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:35:06.544986 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.536645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:60081 (local address 127.27.40.190:40823)
0809 03:35:06.536720 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.536722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.536953 (+ 231us) server_negotiation.cc:407] Connection header received
0809 03:35:06.537420 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.537421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.537431 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.537486 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:06.538405 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.538896 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.539843 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.540040 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.540116 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.542340 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:06.542342 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:06.542342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:06.542350 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:06.543606 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:06.543966 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:06.543966 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:06.543967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:06.544018 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:06.544222 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:06.544222 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:06.544222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:06.544330 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:06.544421 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.544787 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.544893 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:35:06.545145 16860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.536648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:60081)
0809 03:35:06.536815 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.536856 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.537336 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.537587 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.537591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.538124 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:06.538319 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.538324 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.538978 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.538978 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.539768 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.539770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.540137 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.540138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.540242 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.540741 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:06.540749 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:06.542247 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:06.543685 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:06.543691 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:06.543693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:06.543901 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:06.544075 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:06.544075 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:06.544075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:06.544168 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:06.544429 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:06.544436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:06.544608 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.544734 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.544903 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
I20260809 03:35:06.545742 16858 heartbeater.cc:347] Connected to a master server at 127.27.40.190:40823
I20260809 03:35:06.545846 16858 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:06.546034 16858 heartbeater.cc:511] Master 127.27.40.190:40823 requested a full tablet report, sending...
I20260809 03:35:06.546437 16408 ts_manager.cc:194] Registered new tserver with Master: edfea1f6d780425b8a4813df48f6219d (127.27.40.130:36537)
I20260809 03:35:06.546814 16408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:60081
W20260809 03:35:06.627184 16861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:06.627390 16861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:06.627413 16861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:06.628515 16861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:06.628557 16861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:06.628582 16861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:35:06.630270 16861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:39745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=38417
--tserver_master_addrs=127.27.40.190:40823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:06.630488 16861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:06.630671 16861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:06.635835 16867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:06.635923 16861 server_base.cc:1034] running on GCE node
W20260809 03:35:06.635841 16871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:06.635812 16869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:06.636281 16861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:06.636518 16861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:06.637692 16861 hybrid_clock.cc:648] HybridClock initialized: now 1786246506637686 us; error 35 us; skew 500 ppm
Aug 09 03:35:06 dist-test-slave-gkw7 krb5kdc[15593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246506, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:06.640192 16861 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:35:06.640606 16861 webserver.cc:460] Webserver started at http://127.27.40.131:38417/ using document root <none> and password file <none>
I20260809 03:35:06.640853 16861 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:06.640946 16861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:06.642318 16861 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:06.642906 16881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:06.643086 16861 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:35:06.643195 16861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "008e0dbe29434159a95bf1656b084d1c"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "d578c1ec171b8f3e84a5da624bfbb7e4"
server_key_iv: "1108c452f06efb22381b82d2391d099a"
server_key_version: "encryptionkey@0"
I20260809 03:35:06.643505 16861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:06.661870 16861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:06.663787 16861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:06.663898 16861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:06.664072 16861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:06.664400 16861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:06.664458 16861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:06.664500 16861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:06.664527 16861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:06.670679 16861 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:39745
I20260809 03:35:06.670742 17050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:39745 every 8 connection(s)
I20260809 03:35:06.671121 16861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
Aug 09 03:35:06 dist-test-slave-gkw7 krb5kdc[15593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246506, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:06.680512 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.671884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:50843 (local address 127.27.40.190:40823)
0809 03:35:06.671931 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.671933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.672119 (+ 186us) server_negotiation.cc:407] Connection header received
0809 03:35:06.672603 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.672604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.672626 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.672691 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:06.673755 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.674309 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.675265 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.675452 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.675510 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.677725 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:06.677726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:06.677727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:06.677736 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:06.679294 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:06.679654 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:06.679655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:06.679655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:06.679706 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:06.679961 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:06.679962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:06.679962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:06.680094 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:06.680158 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.680382 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.680449 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 03:35:06.680665 17053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.671866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:50843)
0809 03:35:06.672014 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.672048 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.672522 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.672778 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.672782 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.673418 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:06.673674 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.673680 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.674384 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.674385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.675182 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.675183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.675530 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.675531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.675640 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.676136 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:06.676144 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:06.677570 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:06.679364 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:06.679370 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:06.679372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:06.679583 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:06.679767 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:06.679767 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:06.679768 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:06.679875 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:06.680158 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:06.680162 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:06.680271 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.680352 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.680465 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":87,"threads_started":1}
I20260809 03:35:06.680781 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 16861
I20260809 03:35:06.681417 17051 heartbeater.cc:347] Connected to a master server at 127.27.40.190:40823
I20260809 03:35:06.681555 17051 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:06.681746 17051 heartbeater.cc:511] Master 127.27.40.190:40823 requested a full tablet report, sending...
I20260809 03:35:06.682200 16408 ts_manager.cc:194] Registered new tserver with Master: 008e0dbe29434159a95bf1656b084d1c (127.27.40.131:39745)
I20260809 03:35:06.682616 16408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:50843
I20260809 03:35:06.687261 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.681233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39372 (local address 127.27.40.190:40823)
0809 03:35:06.681306 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.681307 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.681468 (+ 161us) server_negotiation.cc:407] Connection header received
0809 03:35:06.681578 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.681579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.681589 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.681631 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:06.682379 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.682895 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.683663 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.683841 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.683900 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.685033 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:06.685034 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:06.685034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:06.685040 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:06.686318 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:06.686666 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:06.686666 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:06.686667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:06.686716 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:06.686878 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:06.686879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:06.686879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:06.686969 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:06.687080 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.687137 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.687192 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 03:35:06.687979 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:35:06.690500 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.688321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39384 (local address 127.27.40.190:40823)
0809 03:35:06.688357 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.688359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.688486 (+ 127us) server_negotiation.cc:407] Connection header received
0809 03:35:06.688531 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.688532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.688559 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.688617 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:06.688721 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.689142 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.689901 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.690070 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.690122 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.690259 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:06.690346 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:06.690404 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.690442 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.690456 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:35:06.690778 16408 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:39384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:35:06.692379 16408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:06.694579 16584 tablet_service.cc:1467] Processing CreateTablet for tablet 83fc6e38e9fd46c58c0a7a63ccc279a9 (DEFAULT_TABLE table=test-table [id=10f4673af0ab4d55833900f91a5cad89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:06.694821 16584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83fc6e38e9fd46c58c0a7a63ccc279a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:06.696432 16529 tablet_bootstrap.cc:492] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599: Bootstrap starting.
I20260809 03:35:06.696801 16529 tablet_bootstrap.cc:654] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:06.697422 16529 tablet_bootstrap.cc:492] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599: No bootstrap required, opened a new log
I20260809 03:35:06.697471 16529 ts_tablet_manager.cc:1397] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:06.697928 16529 raft_consensus.cc:348] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } }
I20260809 03:35:06.698026 16529 raft_consensus.cc:374] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:06.698100 16529 raft_consensus.cc:729] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 938ccc8f4b284b849441e438de98f599, State: Initialized, Role: FOLLOWER
I20260809 03:35:06.698153 16529 consensus_queue.cc:260] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } }
I20260809 03:35:06.698252 16529 ts_tablet_manager.cc:1428] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:06.698602 16686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.694345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:39745 (local address 127.0.0.1:39704)
0809 03:35:06.694401 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.694413 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.694450 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.694935 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.694936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.694938 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:06.694984 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.694985 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.696307 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.696308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.697402 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.697404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.698360 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.698361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.698437 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.698461 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.698495 (+ 34us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.698537 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 03:35:06.698994 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.694408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39704 (local address 127.27.40.131:39745)
0809 03:35:06.694747 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.694752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.694762 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:35:06.694803 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.694805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.694821 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.694910 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:06.695173 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.696193 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.697491 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.698252 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.698800 (+ 548us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.698876 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.698898 (+ 22us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.698932 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":104,"threads_started":1}
I20260809 03:35:06.699689 16952 tablet_service.cc:1467] Processing CreateTablet for tablet 83fc6e38e9fd46c58c0a7a63ccc279a9 (DEFAULT_TABLE table=test-table [id=10f4673af0ab4d55833900f91a5cad89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:06.700093 16952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83fc6e38e9fd46c58c0a7a63ccc279a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:06.700201 17057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.694449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:36537 (local address 127.0.0.1:48368)
0809 03:35:06.694548 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.694556 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.694599 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.696730 (+ 2131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.696731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.696733 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:06.696771 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.696772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.698100 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.698101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.699072 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.699076 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.700005 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.700006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.700073 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.700079 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.700122 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.700153 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":61,"threads_started":1}
I20260809 03:35:06.700258 17058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.694643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48368 (local address 127.27.40.130:36537)
0809 03:35:06.696528 (+ 1885us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.696533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.696543 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:35:06.696600 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.696603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.696620 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.696712 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:06.696846 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.697973 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.699182 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.699892 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.699999 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.700103 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.700135 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.700176 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1843,"thread_start_us":117,"threads_started":1}
I20260809 03:35:06.700966 16772 tablet_service.cc:1467] Processing CreateTablet for tablet 83fc6e38e9fd46c58c0a7a63ccc279a9 (DEFAULT_TABLE table=test-table [id=10f4673af0ab4d55833900f91a5cad89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:06.701301 16772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83fc6e38e9fd46c58c0a7a63ccc279a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:06.703788 17061 tablet_bootstrap.cc:492] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c: Bootstrap starting.
I20260809 03:35:06.704401 17061 tablet_bootstrap.cc:654] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:06.704674 17062 tablet_bootstrap.cc:492] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d: Bootstrap starting.
I20260809 03:35:06.704981 17061 log.cc:824] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:06.705310 17062 tablet_bootstrap.cc:654] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:06.705747 17061 tablet_bootstrap.cc:492] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c: No bootstrap required, opened a new log
I20260809 03:35:06.705832 17061 ts_tablet_manager.cc:1397] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:35:06.705904 17062 log.cc:824] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:06.706681 17062 tablet_bootstrap.cc:492] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d: No bootstrap required, opened a new log
I20260809 03:35:06.706799 17062 ts_tablet_manager.cc:1397] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 03:35:06.707435 17061 raft_consensus.cc:348] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } }
I20260809 03:35:06.707585 17061 raft_consensus.cc:374] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:06.707633 17061 raft_consensus.cc:729] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008e0dbe29434159a95bf1656b084d1c, State: Initialized, Role: FOLLOWER
I20260809 03:35:06.707757 17061 consensus_queue.cc:260] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } }
I20260809 03:35:06.708061 17061 ts_tablet_manager.cc:1428] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 03:35:06.708112 17051 heartbeater.cc:503] Master 127.27.40.190:40823 was elected leader, sending a full tablet report...
I20260809 03:35:06.708374 17062 raft_consensus.cc:348] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } }
I20260809 03:35:06.708536 17062 raft_consensus.cc:374] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:06.708606 17062 raft_consensus.cc:729] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edfea1f6d780425b8a4813df48f6219d, State: Initialized, Role: FOLLOWER
I20260809 03:35:06.708729 17062 consensus_queue.cc:260] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } }
I20260809 03:35:06.709018 17062 ts_tablet_manager.cc:1428] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 03:35:06.709021 16858 heartbeater.cc:503] Master 127.27.40.190:40823 was elected leader, sending a full tablet report...
W20260809 03:35:06.786546 16859 tablet.cc:2369] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:35:06.828610 17067 raft_consensus.cc:482] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:06.828754 17067 raft_consensus.cc:504] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } }
I20260809 03:35:06.829212 17067 leader_election.cc:288] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008e0dbe29434159a95bf1656b084d1c (127.27.40.131:39745), 938ccc8f4b284b849441e438de98f599 (127.27.40.129:36095)
I20260809 03:35:06.832403 16860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.829380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:39745 (local address 127.27.40.130:47093)
0809 03:35:06.829448 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.829462 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.829546 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.829745 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.829746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.829748 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:06.829826 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.829829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.830662 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.830663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.831474 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.831476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.832140 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.832141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.832224 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.832227 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.832287 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.832328 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 03:35:06.832403 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.829480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:47093 (local address 127.27.40.131:39745)
0809 03:35:06.829525 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.829527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.829537 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:35:06.829619 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.829620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.829630 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.829724 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:06.829948 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.830584 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.831559 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.832050 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.832141 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.832151 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.832306 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.832342 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 03:35:06.832703 16688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.829489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:44931 (local address 127.27.40.129:36095)
0809 03:35:06.829559 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.829561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.829788 (+ 227us) server_negotiation.cc:407] Connection header received
0809 03:35:06.829848 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.829849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.829878 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.829945 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:06.830122 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.830770 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.831806 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.832332 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.832422 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.832503 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.832588 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.832634 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 03:35:06.832729 16983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83fc6e38e9fd46c58c0a7a63ccc279a9" candidate_uuid: "edfea1f6d780425b8a4813df48f6219d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008e0dbe29434159a95bf1656b084d1c" is_pre_election: true
I20260809 03:35:06.832885 16983 raft_consensus.cc:2454] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate edfea1f6d780425b8a4813df48f6219d in term 0.
I20260809 03:35:06.832942 17068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.829385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:36095 (local address 127.27.40.130:44931)
0809 03:35:06.829694 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.829703 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.829800 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.829965 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.829966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.829968 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:06.830045 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.830047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.830884 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.830885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.831716 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.831718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.832422 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.832423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.832489 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.832493 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.832817 (+ 324us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.832861 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":54,"thread_start_us":172,"threads_started":1}
I20260809 03:35:06.833087 16706 leader_election.cc:302] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008e0dbe29434159a95bf1656b084d1c, edfea1f6d780425b8a4813df48f6219d; no voters:
I20260809 03:35:06.833267 16608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83fc6e38e9fd46c58c0a7a63ccc279a9" candidate_uuid: "edfea1f6d780425b8a4813df48f6219d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "938ccc8f4b284b849441e438de98f599" is_pre_election: true
I20260809 03:35:06.833353 17067 raft_consensus.cc:2781] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:06.833395 17067 raft_consensus.cc:482] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:06.833451 17067 raft_consensus.cc:3037] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:06.833432 16608 raft_consensus.cc:2454] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate edfea1f6d780425b8a4813df48f6219d in term 0.
I20260809 03:35:06.834292 17067 raft_consensus.cc:504] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } }
I20260809 03:35:06.834491 17067 leader_election.cc:288] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [CANDIDATE]: Term 1 election: Requested vote from peers 008e0dbe29434159a95bf1656b084d1c (127.27.40.131:39745), 938ccc8f4b284b849441e438de98f599 (127.27.40.129:36095)
I20260809 03:35:06.834563 16608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83fc6e38e9fd46c58c0a7a63ccc279a9" candidate_uuid: "edfea1f6d780425b8a4813df48f6219d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "938ccc8f4b284b849441e438de98f599"
I20260809 03:35:06.834648 16983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83fc6e38e9fd46c58c0a7a63ccc279a9" candidate_uuid: "edfea1f6d780425b8a4813df48f6219d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008e0dbe29434159a95bf1656b084d1c"
I20260809 03:35:06.834661 16608 raft_consensus.cc:3037] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:06.834728 16983 raft_consensus.cc:3037] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:06.835595 16608 raft_consensus.cc:2454] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate edfea1f6d780425b8a4813df48f6219d in term 1.
I20260809 03:35:06.835595 16983 raft_consensus.cc:2454] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate edfea1f6d780425b8a4813df48f6219d in term 1.
I20260809 03:35:06.835803 16707 leader_election.cc:302] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 938ccc8f4b284b849441e438de98f599, edfea1f6d780425b8a4813df48f6219d; no voters:
I20260809 03:35:06.835904 17067 raft_consensus.cc:2781] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:06.836048 17067 raft_consensus.cc:686] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [term 1 LEADER]: Becoming Leader. State: Replica: edfea1f6d780425b8a4813df48f6219d, State: Running, Role: LEADER
I20260809 03:35:06.836151 17067 consensus_queue.cc:237] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } }
I20260809 03:35:06.837011 16408 catalog_manager.cc:5495] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d reported cstate change: term changed from 0 to 1, leader changed from <none> to edfea1f6d780425b8a4813df48f6219d (127.27.40.130). New cstate: current_term: 1 leader_uuid: "edfea1f6d780425b8a4813df48f6219d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } health_report { overall_health: UNKNOWN } } }
I20260809 03:35:06.843070 17058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.841048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48370 (local address 127.27.40.130:36537)
0809 03:35:06.841085 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.841087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.841093 (+ 6us) server_negotiation.cc:407] Connection header received
0809 03:35:06.841118 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.841119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.841133 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.841193 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:06.841299 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.841810 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.842482 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.842655 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.842702 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.842829 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:06.842911 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:06.842966 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.842995 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.843009 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 03:35:06.844666 16983 raft_consensus.cc:1261] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c [term 1 FOLLOWER]: Refusing update from remote peer edfea1f6d780425b8a4813df48f6219d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:06.844715 16608 raft_consensus.cc:1261] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Refusing update from remote peer edfea1f6d780425b8a4813df48f6219d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:06.845026 17069 consensus_queue.cc:1035] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [LEADER]: Connected to new peer: Peer: permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:06.845155 17067 consensus_queue.cc:1035] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [LEADER]: Connected to new peer: Peer: permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:06.848018 17074 mvcc.cc:205] Tried to move back new op lower bound from 7316465692033499136 to 7316465692001464320. Current Snapshot: MvccSnapshot[applied={T|T < 7316465692033499136 or (T in {7316465692033499136})}]
I20260809 03:35:06.849417 16408 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:39384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:06.849495 16408 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:39384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:06.850250 16408 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: Sending DeleteTablet for 3 replicas of tablet 83fc6e38e9fd46c58c0a7a63ccc279a9
I20260809 03:35:06.850554 16952 tablet_service.cc:1514] Processing DeleteTablet for tablet 83fc6e38e9fd46c58c0a7a63ccc279a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:06 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:39704
I20260809 03:35:06.850581 16584 tablet_service.cc:1514] Processing DeleteTablet for tablet 83fc6e38e9fd46c58c0a7a63ccc279a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:06 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:38180
I20260809 03:35:06.850606 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 16460
I20260809 03:35:06.850665 16689 tablet_replica.cc:330] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599: stopping tablet replica
I20260809 03:35:06.850672 16772 tablet_service.cc:1514] Processing DeleteTablet for tablet 83fc6e38e9fd46c58c0a7a63ccc279a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:06 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:48368
I20260809 03:35:06.850715 16689 raft_consensus.cc:2229] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:06.850771 16689 raft_consensus.cc:2258] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:06.861848 17084 tablet_replica.cc:330] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c: stopping tablet replica
I20260809 03:35:06.862030 17084 raft_consensus.cc:2229] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:06.862169 17084 raft_consensus.cc:2258] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:06.863436 17084 ts_tablet_manager.cc:1905] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:06.863543 17086 tablet_replica.cc:330] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d: stopping tablet replica
I20260809 03:35:06.863710 17086 raft_consensus.cc:2229] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:35:06.863869 17086 raft_consensus.cc:2258] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:06.864384 17086 ts_tablet_manager.cc:1905] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:06.865597 17084 ts_tablet_manager.cc:1918] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:06.865733 17084 log.cc:1196] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/83fc6e38e9fd46c58c0a7a63ccc279a9
I20260809 03:35:06.866106 17086 ts_tablet_manager.cc:1918] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:06.866117 17084 ts_tablet_manager.cc:1939] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 008e0dbe29434159a95bf1656b084d1c: Deleting consensus metadata
I20260809 03:35:06.866268 17086 log.cc:1196] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/83fc6e38e9fd46c58c0a7a63ccc279a9
I20260809 03:35:06.866622 17086 ts_tablet_manager.cc:1939] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P edfea1f6d780425b8a4813df48f6219d: Deleting consensus metadata
I20260809 03:35:06.867084 16384 catalog_manager.cc:4841] TS edfea1f6d780425b8a4813df48f6219d (127.27.40.130:36537): tablet 83fc6e38e9fd46c58c0a7a63ccc279a9 (table test-table [id=10f4673af0ab4d55833900f91a5cad89]) successfully deleted
W20260809 03:35:06.869045 16386 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.40.129:36095 (error 108)
I20260809 03:35:06.869390 16384 catalog_manager.cc:4841] TS 008e0dbe29434159a95bf1656b084d1c (127.27.40.131:39745): tablet 83fc6e38e9fd46c58c0a7a63ccc279a9 (table test-table [id=10f4673af0ab4d55833900f91a5cad89]) successfully deleted
I20260809 03:35:06.869758 17057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.869613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:36095 (local address 127.0.0.1:38186)
0809 03:35:06.869670 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.869715 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:36095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 03:35:06.869851 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 16681
W20260809 03:35:06.869925 16386 catalog_manager.cc:4584] TS 938ccc8f4b284b849441e438de98f599 (127.27.40.129:36095): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 83fc6e38e9fd46c58c0a7a63ccc279a9: Network error: Client connection negotiation failed: client connection to 127.27.40.129:36095: connect: Connection refused (error 111)
I20260809 03:35:06.887514 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 16861
I20260809 03:35:06.894785 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 16361
I20260809 03:35:06.903860 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:40823
--webserver_interface=127.27.40.190
--webserver_port=40167
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:40823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:06.987582 17088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:06.987787 17088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:06.987808 17088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:06.988973 17088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:06.989015 17088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:06.989030 17088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:06.989042 17088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:06.989055 17088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:06.990530 17088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:40823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:40823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=40167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:06.990787 17088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:06.991035 17088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:06.995990 17101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:06.996129 17088 server_base.cc:1034] running on GCE node
W20260809 03:35:06.996033 17104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:06.996021 17100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:06.996510 17088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:06.996837 17088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:06.998000 17088 hybrid_clock.cc:648] HybridClock initialized: now 1786246506997992 us; error 39 us; skew 500 ppm
Aug 09 03:35:06 dist-test-slave-gkw7 krb5kdc[15593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246506, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:07.000476 17088 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:35:07.000945 17088 webserver.cc:460] Webserver started at http://127.27.40.190:40167/ using document root <none> and password file <none>
I20260809 03:35:07.001174 17088 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:07.001255 17088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:07.002674 17088 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:07.003386 17114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:07.003563 17088 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:35:07.003672 17088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "90af873d5eba42bfb25d219be3957225"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "c3a9e75c9e3ae0a7b11f6eae6e92ab39"
server_key_iv: "b4ea70fdb13a8dc69ecd3cfb99b3b429"
server_key_version: "encryptionkey@0"
I20260809 03:35:07.004009 17088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:07.018303 17088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:07.020234 17088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:07.020406 17088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:07.024828 17088 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:40823
I20260809 03:35:07.024864 17189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:40823 every 8 connection(s)
I20260809 03:35:07.025271 17088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:07.027009 17190 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:35:07.027695 17190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: Bootstrap starting.
I20260809 03:35:07.029093 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 17088
I20260809 03:35:07.029393 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:36095
--local_ip_for_outbound_sockets=127.27.40.129
--tserver_master_addrs=127.27.40.190:40823
--webserver_port=40225
--webserver_interface=127.27.40.129
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:35:07.038228 17190 log.cc:824] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:07.040827 17190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:35:07.041210 17190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: Bootstrap complete.
I20260809 03:35:07.043704 17190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } }
I20260809 03:35:07.043946 17190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90af873d5eba42bfb25d219be3957225, State: Initialized, Role: FOLLOWER
I20260809 03:35:07.044133 17190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } }
I20260809 03:35:07.044226 17190 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:07.044301 17190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:07.044361 17190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 2 FOLLOWER]: Advancing to term 3
I20260809 03:35:07.045499 17190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } }
I20260809 03:35:07.045671 17190 leader_election.cc:302] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90af873d5eba42bfb25d219be3957225; no voters:
I20260809 03:35:07.045912 17190 leader_election.cc:288] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 03:35:07.045962 17195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 03:35:07.046149 17195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [term 3 LEADER]: Becoming Leader. State: Replica: 90af873d5eba42bfb25d219be3957225, State: Running, Role: LEADER
I20260809 03:35:07.046219 17190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:07.046584 17195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } }
I20260809 03:35:07.046918 17195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90af873d5eba42bfb25d219be3957225. Latest consensus state: current_term: 3 leader_uuid: "90af873d5eba42bfb25d219be3957225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } } }
I20260809 03:35:07.047434 17195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:07.047330 17196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "90af873d5eba42bfb25d219be3957225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90af873d5eba42bfb25d219be3957225" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 40823 } } }
I20260809 03:35:07.047904 17196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:07.048415 17219 catalog_manager.cc:1259] Loaded cluster ID: b177eedca0694c48bab4c6600f5ba101
I20260809 03:35:07.048468 17219 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: loading cluster ID for follower catalog manager: success
W20260809 03:35:07.048777 17219 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 90af873d5eba42bfb25d219be3957225: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260809 03:35:07.048825 17219 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260809 03:35:07.048859 17213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:07.049233 17213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:07.049299 17213 catalog_manager.cc:1259] Loaded cluster ID: b177eedca0694c48bab4c6600f5ba101
I20260809 03:35:07.049320 17213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 03:35:07.049438 17213 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe0587d29ad google::LogMessage::Fail() at ??:0
@ 0x7fe0587d588a google::LogMessage::SendToLog() at ??:0
@ 0x7fe0587d24de google::LogMessage::Flush() at ??:0
@ 0x7fe0587d3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe0606f30f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fe0606fc8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe05a6a00d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe05a695954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe05b4f96db start_thread at ??:0
@ 0x7fe0572d061f clone at ??:0
W20260809 03:35:07.150167 17192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:07.150440 17192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:07.150478 17192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:07.152115 17192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:07.152177 17192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:07.152222 17192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:35:07.154458 17192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:36095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=40225
--tserver_master_addrs=127.27.40.190:40823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:07.154721 17192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:07.155016 17192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:07.160640 17254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:07.160624 17251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:07.160650 17252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:07.160983 17192 server_base.cc:1034] running on GCE node
I20260809 03:35:07.161194 17192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:07.161446 17192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:07.162623 17192 hybrid_clock.cc:648] HybridClock initialized: now 1786246507162611 us; error 34 us; skew 500 ppm
Aug 09 03:35:07 dist-test-slave-gkw7 krb5kdc[15593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246507, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:07.165143 17192 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:35:07.165617 17192 webserver.cc:460] Webserver started at http://127.27.40.129:40225/ using document root <none> and password file <none>
I20260809 03:35:07.165879 17192 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:07.165925 17192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:07.167387 17192 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:07.168160 17267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:07.168356 17192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:07.168478 17192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "938ccc8f4b284b849441e438de98f599"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "79a59517dc60d678eaf4501d7fe2952b"
server_key_iv: "081b358f7fc2c4556d13a68d811c6b8c"
server_key_version: "encryptionkey@0"
I20260809 03:35:07.168829 17192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:07.192153 17192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:07.194584 17192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:07.194770 17192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:07.195051 17192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:07.195485 17277 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 03:35:07.197628 17192 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 03:35:07.197698 17192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 03:35:07.197760 17192 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 03:35:07.198397 17192 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 03:35:07.198453 17192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 03:35:07.198535 17277 tablet_bootstrap.cc:492] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599: Bootstrap starting.
I20260809 03:35:07.205871 17192 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:36095
I20260809 03:35:07.205984 17427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:36095 every 8 connection(s)
I20260809 03:35:07.206298 17192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:07.207047 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 17192
I20260809 03:35:07.207335 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:36537
--local_ip_for_outbound_sockets=127.27.40.130
--tserver_master_addrs=127.27.40.190:40823
--webserver_port=39585
--webserver_interface=127.27.40.130
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:35:07.209836 17277 log.cc:824] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:07.212863 17277 tablet_bootstrap.cc:492] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:35:07.213178 17277 tablet_bootstrap.cc:492] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599: Bootstrap complete.
I20260809 03:35:07.214552 17277 ts_tablet_manager.cc:1397] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.011s
I20260809 03:35:07.216315 17277 raft_consensus.cc:348] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } }
I20260809 03:35:07.216718 17277 raft_consensus.cc:729] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 938ccc8f4b284b849441e438de98f599, State: Initialized, Role: FOLLOWER
I20260809 03:35:07.216915 17277 consensus_queue.cc:260] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } }
I20260809 03:35:07.217211 17277 ts_tablet_manager.cc:1428] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
W20260809 03:35:07.272416 17431 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:07.207078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:43995)
0809 03:35:07.209356 (+ 2278us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:07.209405 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:07.210097 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:07.272190 (+ 62093us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":2232,"thread_start_us":100,"threads_started":1}
W20260809 03:35:07.272775 17428 heartbeater.cc:644] Failed to heartbeat to 127.27.40.190:40823 (0 consecutive failures): Network error: Failed to ping master at 127.27.40.190:40823: Client connection negotiation failed: client connection to 127.27.40.190:40823: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 03:35:07.273187 17431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:07.273037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:41749)
0809 03:35:07.273099 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:07.273155 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 03:35:07.273677 17431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:07.273544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:49903)
0809 03:35:07.273608 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:07.273637 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 03:35:07.273887 17428 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:35:07.297936 17433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:07.298128 17433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:07.298151 17433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:07.299285 17433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:07.299326 17433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:07.299351 17433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:35:07.300757 17433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:36537
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=39585
--tserver_master_addrs=127.27.40.190:40823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:07.301007 17433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:07.301239 17433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:07.306330 17447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:07.306340 17445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:07.306370 17444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:07.306708 17433 server_base.cc:1034] running on GCE node
I20260809 03:35:07.306939 17433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:07.307183 17433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:07.308350 17433 hybrid_clock.cc:648] HybridClock initialized: now 1786246507308348 us; error 40 us; skew 500 ppm
Aug 09 03:35:07 dist-test-slave-gkw7 krb5kdc[15593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246507, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:07.310973 17433 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:35:07.311452 17433 webserver.cc:460] Webserver started at http://127.27.40.130:39585/ using document root <none> and password file <none>
I20260809 03:35:07.311688 17433 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:07.311765 17433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:07.313064 17433 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:07.313724 17455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:07.313920 17433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:07.314023 17433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "edfea1f6d780425b8a4813df48f6219d"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "abe2abaaeeaa7e0b353250e3f04d038d"
server_key_iv: "92fac19e97afc7d95525113807574716"
server_key_version: "encryptionkey@0"
I20260809 03:35:07.314390 17433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:07.328251 17433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:07.330559 17433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:07.330727 17433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:07.330963 17433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:07.331286 17433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:07.331342 17433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:07.331384 17433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:07.331411 17433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:07.337566 17433 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:36537
I20260809 03:35:07.337608 17618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:36537 every 8 connection(s)
I20260809 03:35:07.337993 17433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:35:07.339021 17622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:07.338632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:44793)
0809 03:35:07.338793 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:07.338867 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
W20260809 03:35:07.339463 17619 heartbeater.cc:644] Failed to heartbeat to 127.27.40.190:40823 (0 consecutive failures): Network error: Failed to ping master at 127.27.40.190:40823: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
I20260809 03:35:07.339781 17622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:07.339689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:32943)
0809 03:35:07.339737 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:07.339763 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 03:35:07.340174 17622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:07.339991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:48859)
0809 03:35:07.340124 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:07.340153 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260809 03:35:07.340361 17619 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 03:35:07.348078 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 17433
I20260809 03:35:07.348284 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:39745
--local_ip_for_outbound_sockets=127.27.40.131
--tserver_master_addrs=127.27.40.190:40823
--webserver_port=38417
--webserver_interface=127.27.40.131
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:44691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:07.431509 17623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:07.431752 17623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:07.431788 17623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:07.432910 17623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:07.432976 17623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:07.433020 17623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:35:07.434518 17623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:44691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:39745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=38417
--tserver_master_addrs=127.27.40.190:40823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:07.434780 17623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:07.435017 17623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:07.440212 17631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:07.440207 17629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:07.440191 17628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:07.440354 17623 server_base.cc:1034] running on GCE node
I20260809 03:35:07.440593 17623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:07.440837 17623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:07.441998 17623 hybrid_clock.cc:648] HybridClock initialized: now 1786246507441993 us; error 37 us; skew 500 ppm
Aug 09 03:35:07 dist-test-slave-gkw7 krb5kdc[15593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246507, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:07.444579 17623 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:35:07.445000 17623 webserver.cc:460] Webserver started at http://127.27.40.131:38417/ using document root <none> and password file <none>
I20260809 03:35:07.445250 17623 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:07.445319 17623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:07.446676 17623 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:07.447309 17639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:07.447484 17623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:35:07.447587 17623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "008e0dbe29434159a95bf1656b084d1c"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-gkw7"
server_key: "d578c1ec171b8f3e84a5da624bfbb7e4"
server_key_iv: "1108c452f06efb22381b82d2391d099a"
server_key_version: "encryptionkey@0"
I20260809 03:35:07.447923 17623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:07.458783 17623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:07.461004 17623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:07.461159 17623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:07.461385 17623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:07.461738 17623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:07.461813 17623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:07.461855 17623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:07.461885 17623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:07.467672 17623 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:39745
I20260809 03:35:07.467765 17805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:39745 every 8 connection(s)
I20260809 03:35:07.468111 17623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:07.469228 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:07.468773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:43237)
0809 03:35:07.468979 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:07.469081 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":76,"threads_started":1}
W20260809 03:35:07.469710 17806 heartbeater.cc:644] Failed to heartbeat to 127.27.40.190:40823 (0 consecutive failures): Network error: Failed to ping master at 127.27.40.190:40823: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
I20260809 03:35:07.469978 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:07.469892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:49801)
0809 03:35:07.469936 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:07.469962 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 03:35:07.470234 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:07.470174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:41331)
0809 03:35:07.470199 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:07.470220 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260809 03:35:07.470348 17806 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 03:35:07.474063 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 17623
I20260809 03:35:08.274453 17811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.274199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:58201)
0809 03:35:08.274349 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:08.274389 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":72,"threads_started":1}
I20260809 03:35:08.341017 17812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.340750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:50463)
0809 03:35:08.340890 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:08.340939 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260809 03:35:08.470952 17813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.470689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:50023)
0809 03:35:08.470851 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:08.470891 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":71,"threads_started":1}
I20260809 03:35:08.638929 17815 raft_consensus.cc:482] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:08.639047 17815 raft_consensus.cc:504] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } }
I20260809 03:35:08.639375 17815 leader_election.cc:288] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 008e0dbe29434159a95bf1656b084d1c (127.27.40.131:39745), edfea1f6d780425b8a4813df48f6219d (127.27.40.130:36537)
Aug 09 03:35:08 dist-test-slave-gkw7 krb5kdc[15593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246507, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.131@KRBTEST.COM
Aug 09 03:35:08 dist-test-slave-gkw7 krb5kdc[15593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246507, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.130@KRBTEST.COM
I20260809 03:35:08.653136 17811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.639561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:39745 (local address 127.27.40.129:55131)
0809 03:35:08.639649 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:08.639665 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:08.639740 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:08.640870 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:08.640873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:08.641429 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:08.641738 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:08.641745 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.643691 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.643692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:08.645033 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:08.645036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.646032 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.646033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:08.646174 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:08.646939 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:08.646951 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:08.648952 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:08.651374 (+ 2422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:08.651381 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:08.651383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:08.651632 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:08.651939 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:08.651940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:08.651941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:08.652060 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:08.652544 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:08.652553 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:08.652737 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 03:35:08.652862 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 03:35:08.653009 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":93}
I20260809 03:35:08.653220 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.639773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:55131 (local address 127.27.40.131:39745)
0809 03:35:08.639905 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:08.639920 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:08.639969 (+ 49us) server_negotiation.cc:407] Connection header received
0809 03:35:08.640570 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:08.640573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:08.640635 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:08.640797 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:08.641925 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.643549 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.645153 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.645916 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.646098 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:08.649096 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:08.649107 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:08.649108 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:08.649120 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:08.651293 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:08.651727 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:08.651731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:08.651732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:08.651852 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:08.652133 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:08.652135 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:08.652135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:08.652451 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:08.652540 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:08.652942 (+ 402us) server_negotiation.cc:299] Negotiation successful
0809 03:35:08.653090 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":60,"threads_started":1}
I20260809 03:35:08.653364 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.639713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:36537 (local address 127.27.40.129:33245)
0809 03:35:08.639851 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:08.639861 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:08.639926 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:08.640870 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:08.640873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:08.641429 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:08.641738 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:08.641745 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.643695 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.643696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:08.645033 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:08.645036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.646030 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.646032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:08.646174 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:08.646946 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:08.646951 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:08.649370 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:08.651831 (+ 2461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:08.651832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:08.651833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:08.652055 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:08.652445 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:08.652446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:08.652446 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:08.652520 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:08.653041 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:08.653042 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:08.653090 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 03:35:08.653154 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 03:35:08.653256 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":62,"thread_start_us":86,"threads_started":1}
I20260809 03:35:08.653750 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.639820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:33245 (local address 127.27.40.130:36537)
0809 03:35:08.639994 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:08.640002 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:08.640031 (+ 29us) server_negotiation.cc:407] Connection header received
0809 03:35:08.640570 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:08.640574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:08.640651 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:08.640797 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:08.641924 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.643553 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.645153 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.645916 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.646063 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:08.649504 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:08.649508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:08.649508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:08.649518 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:08.651714 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:08.652196 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:08.652200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:08.652200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:08.652355 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:08.652603 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:08.652605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:08.652605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:08.652931 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:08.653010 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:08.653495 (+ 485us) server_negotiation.cc:299] Negotiation successful
0809 03:35:08.653669 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":75,"threads_started":1}
W20260809 03:35:08.654176 17268 leader_election.cc:341] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 008e0dbe29434159a95bf1656b084d1c (127.27.40.131:39745): Not found: Tablet not found: 83fc6e38e9fd46c58c0a7a63ccc279a9
W20260809 03:35:08.654507 17268 leader_election.cc:341] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer edfea1f6d780425b8a4813df48f6219d (127.27.40.130:36537): Not found: Tablet not found: 83fc6e38e9fd46c58c0a7a63ccc279a9
I20260809 03:35:08.654560 17268 leader_election.cc:302] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 938ccc8f4b284b849441e438de98f599; no voters: 008e0dbe29434159a95bf1656b084d1c, edfea1f6d780425b8a4813df48f6219d
I20260809 03:35:08.654719 17815 raft_consensus.cc:2726] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:35:09.275134 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:09.274856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:59463)
0809 03:35:09.275010 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:09.275070 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
I20260809 03:35:09.341758 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:09.341507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:41651)
0809 03:35:09.341680 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:09.341713 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":122,"threads_started":1}
I20260809 03:35:09.471697 17821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:09.471460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:55533)
0809 03:35:09.471604 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:09.471656 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260809 03:35:10.262023 17822 raft_consensus.cc:482] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:10.262177 17822 raft_consensus.cc:504] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } }
I20260809 03:35:10.262388 17822 leader_election.cc:288] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 008e0dbe29434159a95bf1656b084d1c (127.27.40.131:39745), edfea1f6d780425b8a4813df48f6219d (127.27.40.130:36537)
W20260809 03:35:10.262877 17268 leader_election.cc:341] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 008e0dbe29434159a95bf1656b084d1c (127.27.40.131:39745): Not found: Tablet not found: 83fc6e38e9fd46c58c0a7a63ccc279a9
W20260809 03:35:10.262950 17268 leader_election.cc:341] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer edfea1f6d780425b8a4813df48f6219d (127.27.40.130:36537): Not found: Tablet not found: 83fc6e38e9fd46c58c0a7a63ccc279a9
I20260809 03:35:10.262967 17268 leader_election.cc:302] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 938ccc8f4b284b849441e438de98f599; no voters: 008e0dbe29434159a95bf1656b084d1c, edfea1f6d780425b8a4813df48f6219d
I20260809 03:35:10.263036 17822 raft_consensus.cc:2726] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:35:10.275792 17823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:10.275569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:38441)
0809 03:35:10.275699 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:10.275748 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260809 03:35:10.342449 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:10.342235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:39523)
0809 03:35:10.342368 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:10.342406 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260809 03:35:10.472347 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:10.472131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:32787)
0809 03:35:10.472263 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:10.472304 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260809 03:35:11.276595 17827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:11.276325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:54797)
0809 03:35:11.276481 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:11.276533 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":65,"threads_started":1}
I20260809 03:35:11.343072 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:11.342872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:57259)
0809 03:35:11.342993 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:11.343032 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260809 03:35:11.473124 17830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:11.472914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:57337)
0809 03:35:11.473042 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:11.473082 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260809 03:35:12.277411 17832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:12.277153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:36007)
0809 03:35:12.277295 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:12.277348 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260809 03:35:12.343778 17833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:12.343542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:56115)
0809 03:35:12.343694 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:12.343733 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":64,"threads_started":1}
I20260809 03:35:12.473845 17834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:12.473637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:44579)
0809 03:35:12.473767 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:12.473804 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
I20260809 03:35:13.027458 17835 raft_consensus.cc:482] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:13.027577 17835 raft_consensus.cc:504] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } }
I20260809 03:35:13.027781 17835 leader_election.cc:288] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 008e0dbe29434159a95bf1656b084d1c (127.27.40.131:39745), edfea1f6d780425b8a4813df48f6219d (127.27.40.130:36537)
W20260809 03:35:13.028168 17268 leader_election.cc:341] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 008e0dbe29434159a95bf1656b084d1c (127.27.40.131:39745): Not found: Tablet not found: 83fc6e38e9fd46c58c0a7a63ccc279a9
W20260809 03:35:13.028246 17268 leader_election.cc:341] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer edfea1f6d780425b8a4813df48f6219d (127.27.40.130:36537): Not found: Tablet not found: 83fc6e38e9fd46c58c0a7a63ccc279a9
I20260809 03:35:13.028263 17268 leader_election.cc:302] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 938ccc8f4b284b849441e438de98f599; no voters: 008e0dbe29434159a95bf1656b084d1c, edfea1f6d780425b8a4813df48f6219d
I20260809 03:35:13.028349 17835 raft_consensus.cc:2726] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:35:13.278158 17836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:13.277895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:60341)
0809 03:35:13.278047 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:13.278093 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260809 03:35:13.344476 17838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:13.344266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:44713)
0809 03:35:13.344399 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:13.344437 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260809 03:35:13.474540 17840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:13.474304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:35139)
0809 03:35:13.474459 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:13.474498 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260809 03:35:14.279171 17842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:14.278913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:45611)
0809 03:35:14.279079 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:14.279122 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260809 03:35:14.345155 17845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:14.344944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:54597)
0809 03:35:14.345069 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:14.345109 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260809 03:35:14.475230 17847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:14.475010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:37505)
0809 03:35:14.475146 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:14.475188 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260809 03:35:15.279903 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:15.279646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:37917)
0809 03:35:15.279794 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:15.279840 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260809 03:35:15.345862 17849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:15.345669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:42153)
0809 03:35:15.345789 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:15.345820 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260809 03:35:15.476073 17850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:15.475837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:44215)
0809 03:35:15.475985 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:15.476025 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":68,"threads_started":1}
I20260809 03:35:16.280793 17852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:16.280516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:54127)
0809 03:35:16.280688 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:16.280724 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260809 03:35:16.346481 17853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:16.346268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:35503)
0809 03:35:16.346404 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:16.346437 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260809 03:35:16.476816 17855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:16.476582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:38321)
0809 03:35:16.476731 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:16.476775 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260809 03:35:17.234418 17856 raft_consensus.cc:482] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:17.234520 17856 raft_consensus.cc:504] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } }
I20260809 03:35:17.234710 17856 leader_election.cc:288] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 008e0dbe29434159a95bf1656b084d1c (127.27.40.131:39745), edfea1f6d780425b8a4813df48f6219d (127.27.40.130:36537)
W20260809 03:35:17.235189 17268 leader_election.cc:341] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 008e0dbe29434159a95bf1656b084d1c (127.27.40.131:39745): Not found: Tablet not found: 83fc6e38e9fd46c58c0a7a63ccc279a9
W20260809 03:35:17.235282 17268 leader_election.cc:341] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer edfea1f6d780425b8a4813df48f6219d (127.27.40.130:36537): Not found: Tablet not found: 83fc6e38e9fd46c58c0a7a63ccc279a9
I20260809 03:35:17.235302 17268 leader_election.cc:302] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 938ccc8f4b284b849441e438de98f599; no voters: 008e0dbe29434159a95bf1656b084d1c, edfea1f6d780425b8a4813df48f6219d
I20260809 03:35:17.235371 17856 raft_consensus.cc:2726] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:35:17.281494 17857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:17.281230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:52305)
0809 03:35:17.281387 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:17.281431 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260809 03:35:17.347187 17859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:17.346957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:60277)
0809 03:35:17.347084 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:17.347125 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260809 03:35:17.477605 17860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:17.477316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:57547)
0809 03:35:17.477463 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:17.477499 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260809 03:35:18.282256 17861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:18.281989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:33835)
0809 03:35:18.282170 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:18.282209 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":68,"threads_started":1}
I20260809 03:35:18.347862 17862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:18.347617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:39395)
0809 03:35:18.347776 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:18.347815 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
I20260809 03:35:18.478411 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:18.478165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:38633)
0809 03:35:18.478325 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:18.478364 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":82,"threads_started":1}
I20260809 03:35:19.282907 17864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:19.282677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:36407)
0809 03:35:19.282815 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:19.282849 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
I20260809 03:35:19.348559 17866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:19.348322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:35915)
0809 03:35:19.348482 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:19.348515 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
I20260809 03:35:19.479171 17868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:19.478908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:42537)
0809 03:35:19.479080 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:19.479124 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":112,"threads_started":1}
I20260809 03:35:20.283627 17870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:20.283391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:33523)
0809 03:35:20.283540 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:20.283578 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260809 03:35:20.349268 17871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:20.349009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:58107)
0809 03:35:20.349185 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:20.349219 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":92,"threads_started":1}
I20260809 03:35:20.479913 17872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:20.479688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:41333)
0809 03:35:20.479823 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:20.479864 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260809 03:35:21.284411 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.284180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:45161)
0809 03:35:21.284324 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:21.284364 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260809 03:35:21.350030 17874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.349780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:33669)
0809 03:35:21.349930 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:21.349976 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260809 03:35:21.480813 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.480500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:43327)
0809 03:35:21.480728 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:21.480768 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":78,"threads_started":1}
I20260809 03:35:21.529321 17877 raft_consensus.cc:482] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:21.529421 17877 raft_consensus.cc:504] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008e0dbe29434159a95bf1656b084d1c" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 39745 } } peers { permanent_uuid: "edfea1f6d780425b8a4813df48f6219d" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 36537 } } peers { permanent_uuid: "938ccc8f4b284b849441e438de98f599" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 36095 } }
I20260809 03:35:21.529691 17877 leader_election.cc:288] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 008e0dbe29434159a95bf1656b084d1c (127.27.40.131:39745), edfea1f6d780425b8a4813df48f6219d (127.27.40.130:36537)
W20260809 03:35:21.530154 17268 leader_election.cc:341] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer edfea1f6d780425b8a4813df48f6219d (127.27.40.130:36537): Not found: Tablet not found: 83fc6e38e9fd46c58c0a7a63ccc279a9
W20260809 03:35:21.530220 17268 leader_election.cc:341] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 008e0dbe29434159a95bf1656b084d1c (127.27.40.131:39745): Not found: Tablet not found: 83fc6e38e9fd46c58c0a7a63ccc279a9
I20260809 03:35:21.530238 17268 leader_election.cc:302] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 938ccc8f4b284b849441e438de98f599; no voters: 008e0dbe29434159a95bf1656b084d1c, edfea1f6d780425b8a4813df48f6219d
I20260809 03:35:21.530314 17877 raft_consensus.cc:2726] T 83fc6e38e9fd46c58c0a7a63ccc279a9 P 938ccc8f4b284b849441e438de98f599 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:35:22.285171 17879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.284930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.129:37337)
0809 03:35:22.285088 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:22.285125 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":61,"threads_started":1}
I20260809 03:35:22.350796 17882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.350551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.130:49039)
0809 03:35:22.350707 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:22.350748 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":59,"threads_started":1}
I20260809 03:35:22.481796 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.481376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:40823 (local address 127.27.40.131:46075)
0809 03:35:22.481700 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:22.481740 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":89,"threads_started":1}
I20260809 03:35:22.484338 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 17192
I20260809 03:35:22.500135 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 17433
I20260809 03:35:22.510633 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 17623
2026-08-09T03:35:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18561 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:35:22 dist-test-slave-gkw7 krb5kdc[17888](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:35:22 dist-test-slave-gkw7 krb5kdc[17888](info): set up 2 sockets
Aug 09 03:35:22 dist-test-slave-gkw7 krb5kdc[17888](info): commencing operation
krb5kdc: starting...
W20260809 03:35:23.799860 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:35:23 dist-test-slave-gkw7 krb5kdc[17888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:35:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:35:23.920259 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:37221
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:34683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:37221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:24.003656 17906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:24.003916 17906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:24.003952 17906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:24.005120 17906 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 03:35:24.005184 17906 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 03:35:24.005208 17906 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 03:35:24.005228 17906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:24.005259 17906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:24.005290 17906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:24.005316 17906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:24.005345 17906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:24.006980 17906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:34683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:37221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:37221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:24.007277 17906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:24.007517 17906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:24.012696 17913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:24.012816 17906 server_base.cc:1034] running on GCE node
W20260809 03:35:24.012728 17915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:24.012765 17911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:24.013240 17906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:24.013504 17906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:24.014653 17906 hybrid_clock.cc:648] HybridClock initialized: now 1786246524014641 us; error 30 us; skew 500 ppm
Aug 09 03:35:24 dist-test-slave-gkw7 krb5kdc[17888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246524, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:24.017248 17906 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:35:24.019414 17906 webserver.cc:460] Webserver started at http://127.27.40.190:37593/ using document root <none> and password file <none>
I20260809 03:35:24.019665 17906 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:24.019706 17906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:24.019795 17906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:24.020833 17906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "9a93316b6a59451db26e3fde483c11ea"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "6b83e9db128fb8140311fd6b722dc023"
server_key_iv: "b25c7bcf60d961d1837410695f05b603"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.021134 17906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "9a93316b6a59451db26e3fde483c11ea"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "6b83e9db128fb8140311fd6b722dc023"
server_key_iv: "b25c7bcf60d961d1837410695f05b603"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.022514 17906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.005s sys 0.000s
I20260809 03:35:24.023205 17926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:24.023379 17906 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:35:24.023489 17906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "9a93316b6a59451db26e3fde483c11ea"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "6b83e9db128fb8140311fd6b722dc023"
server_key_iv: "b25c7bcf60d961d1837410695f05b603"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.023566 17906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:24.057330 17906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:24.059800 17906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:24.060002 17906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:24.064218 17906 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:37221
I20260809 03:35:24.064255 17999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:37221 every 8 connection(s)
I20260809 03:35:24.064646 17906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:24.065403 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 17906
I20260809 03:35:24.065551 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:35:24.065761 18000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:24.065802 27810 external_mini_cluster.cc:1428] Setting key 41a9c3f138a5923e293bd7415807ea09
I20260809 03:35:24.069239 18000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Bootstrap starting.
I20260809 03:35:24.070010 18000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:24.070344 18000 log.cc:824] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:24.071298 18000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: No bootstrap required, opened a new log
Aug 09 03:35:24 dist-test-slave-gkw7 krb5kdc[17888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:24.073593 18000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } }
I20260809 03:35:24.073755 18000 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:24.073805 18000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a93316b6a59451db26e3fde483c11ea, State: Initialized, Role: FOLLOWER
I20260809 03:35:24.073973 18000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } }
I20260809 03:35:24.074074 18000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:24.074126 18000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:24.074182 18000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:24.074730 18000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } }
I20260809 03:35:24.074860 18000 leader_election.cc:302] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a93316b6a59451db26e3fde483c11ea; no voters:
I20260809 03:35:24.075068 18000 leader_election.cc:288] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:24.075102 18007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:24.075222 18007 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 1 LEADER]: Becoming Leader. State: Replica: 9a93316b6a59451db26e3fde483c11ea, State: Running, Role: LEADER
I20260809 03:35:24.075366 18000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:24.075366 18007 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } }
I20260809 03:35:24.075834 18013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a93316b6a59451db26e3fde483c11ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } } }
I20260809 03:35:24.076073 18013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:24.075981 18008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a93316b6a59451db26e3fde483c11ea. Latest consensus state: current_term: 1 leader_uuid: "9a93316b6a59451db26e3fde483c11ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } } }
I20260809 03:35:24.076561 18008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:24.076663 18021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:24.077003 18005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.066349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41232 (local address 127.27.40.190:37221)
0809 03:35:24.066534 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:24.066543 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:24.066569 (+ 26us) server_negotiation.cc:407] Connection header received
0809 03:35:24.066967 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:24.066973 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:24.067017 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:24.067123 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:24.067955 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.068802 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.069906 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.070362 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.070484 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.073088 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:24.073092 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:24.073092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:24.073105 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:24.074998 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:24.075422 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:24.075426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:24.075426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:24.075520 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:24.075721 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:24.075721 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:24.075722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:24.076037 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:24.076197 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:24.076645 (+ 448us) server_negotiation.cc:299] Negotiation successful
0809 03:35:24.076805 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260809 03:35:24.077212 18021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:24.078902 18021 catalog_manager.cc:1347] Generated new cluster ID: 509f493c6db64e56a5166d4a2268684a
I20260809 03:35:24.078970 18021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:24.095412 18021 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:24.096006 18021 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:24.099170 18021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Generated new TSK 0
I20260809 03:35:24.099371 18021 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:35:24.129689 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37221
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:34683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:24.213040 18040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:24.213286 18040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:24.213326 18040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:24.214476 18040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:24.214547 18040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:24.214589 18040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:35:24.216085 18040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:34683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:24.216353 18040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:24.216586 18040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:24.221910 18045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:24.221925 18046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:24.222111 18049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:24.222064 18040 server_base.cc:1034] running on GCE node
I20260809 03:35:24.222529 18040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:24.222786 18040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:24.224004 18040 hybrid_clock.cc:648] HybridClock initialized: now 1786246524223988 us; error 42 us; skew 500 ppm
Aug 09 03:35:24 dist-test-slave-gkw7 krb5kdc[17888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246524, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:24.226979 18040 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:35:24.227535 18040 webserver.cc:460] Webserver started at http://127.27.40.129:41367/ using document root <none> and password file <none>
I20260809 03:35:24.227746 18040 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:24.227790 18040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:24.227881 18040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:24.228897 18040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "328a09530d3d46db8d76405f32d1580b"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "211b307adc91b9bcb641cb3d9a47632c"
server_key_iv: "398f2f86002aa916f01815a981975f33"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.229207 18040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "328a09530d3d46db8d76405f32d1580b"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "211b307adc91b9bcb641cb3d9a47632c"
server_key_iv: "398f2f86002aa916f01815a981975f33"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.230623 18040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:35:24.231375 18056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:24.231606 18040 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 03:35:24.231700 18040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "328a09530d3d46db8d76405f32d1580b"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "211b307adc91b9bcb641cb3d9a47632c"
server_key_iv: "398f2f86002aa916f01815a981975f33"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.231753 18040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:24.248003 18040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:24.250733 18040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:24.250859 18040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:24.251063 18040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:24.251415 18040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:24.251452 18040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:24.251498 18040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:24.251524 18040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:24.257699 18040 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:34973
I20260809 03:35:24.257771 18226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:34973 every 8 connection(s)
I20260809 03:35:24.258111 18040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
Aug 09 03:35:24 dist-test-slave-gkw7 krb5kdc[17888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246524, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:24.265231 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 18040
I20260809 03:35:24.265354 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:35:24.265645 27810 external_mini_cluster.cc:1428] Setting key 0b311a50f6bb93969c6be117b06d4906
I20260809 03:35:24.268406 18005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.258911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:45223 (local address 127.27.40.190:37221)
0809 03:35:24.258987 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:24.258990 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:24.259227 (+ 237us) server_negotiation.cc:407] Connection header received
0809 03:35:24.259714 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:24.259715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:24.259727 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:24.259781 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:24.260823 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.261403 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.262535 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.262758 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.262817 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.265106 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:24.265113 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:24.265113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:24.265120 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:24.266889 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:24.267289 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:24.267290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:24.267291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:24.267358 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:24.267642 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:24.267643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:24.267643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:24.267801 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:24.267873 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:24.268158 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 03:35:24.268257 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 03:35:24.268507 18234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.258896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:45223)
0809 03:35:24.259095 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:24.259133 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:24.259630 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:24.259857 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:24.259860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:24.260492 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:24.260702 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.260707 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.261521 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.261522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.262443 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.262445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.262840 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.262841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.262942 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.263444 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:24.263452 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:24.264960 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:24.266971 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:24.266979 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:24.266980 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:24.267212 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:24.267441 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:24.267442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:24.267442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:24.267569 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:24.267883 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:24.267889 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:24.268013 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 03:35:24.268114 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 03:35:24.268289 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":110,"threads_started":1}
I20260809 03:35:24.269049 18227 heartbeater.cc:347] Connected to a master server at 127.27.40.190:37221
I20260809 03:35:24.269176 18227 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:24.269378 18227 heartbeater.cc:511] Master 127.27.40.190:37221 requested a full tablet report, sending...
I20260809 03:35:24.270010 17948 ts_manager.cc:194] Registered new tserver with Master: 328a09530d3d46db8d76405f32d1580b (127.27.40.129:34973)
I20260809 03:35:24.270745 17948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:45223
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:35:24.294384 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37221
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:34683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:24.381139 18237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:24.381356 18237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:24.381379 18237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:24.382622 18237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:24.382668 18237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:24.382696 18237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:35:24.384158 18237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:34683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:24.384411 18237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:24.384656 18237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:24.389806 18245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:24.389822 18244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:24.389822 18250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:24.390161 18237 server_base.cc:1034] running on GCE node
I20260809 03:35:24.390429 18237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:24.390686 18237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:24.391862 18237 hybrid_clock.cc:648] HybridClock initialized: now 1786246524391849 us; error 29 us; skew 500 ppm
Aug 09 03:35:24 dist-test-slave-gkw7 krb5kdc[17888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246524, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:24.394449 18237 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:35:24.394897 18237 webserver.cc:460] Webserver started at http://127.27.40.130:39503/ using document root <none> and password file <none>
I20260809 03:35:24.395133 18237 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:24.395203 18237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:24.395361 18237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:24.396624 18237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "d959522beac347448ef72ef2860cc31a"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "4c231cad4f3cbb4a31c98ca1775563fe"
server_key_iv: "e36fef61f68f8eab5998f9c514196fb6"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.396930 18237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "d959522beac347448ef72ef2860cc31a"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "4c231cad4f3cbb4a31c98ca1775563fe"
server_key_iv: "e36fef61f68f8eab5998f9c514196fb6"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.398360 18237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:35:24.399081 18258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:24.399281 18237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 03:35:24.399381 18237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "d959522beac347448ef72ef2860cc31a"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "4c231cad4f3cbb4a31c98ca1775563fe"
server_key_iv: "e36fef61f68f8eab5998f9c514196fb6"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.399473 18237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:24.413774 18237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:24.415794 18237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:24.415968 18237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:24.416204 18237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:24.416519 18237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:24.416575 18237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:24.416625 18237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:24.416654 18237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:24.422729 18237 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:40565
I20260809 03:35:24.422781 18431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:40565 every 8 connection(s)
I20260809 03:35:24.423096 18237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
Aug 09 03:35:24 dist-test-slave-gkw7 krb5kdc[17888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246524, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:24.430313 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 18237
I20260809 03:35:24.430434 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:35:24.430698 27810 external_mini_cluster.cc:1428] Setting key 66093687651691601be3a68b5d7f49d4
I20260809 03:35:24.433398 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.423789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:43821)
0809 03:35:24.423960 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:24.424002 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:24.424481 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:24.424715 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:24.424718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:24.425310 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:24.425517 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.425523 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.426428 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.426429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.427293 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.427295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.427699 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.427700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.427804 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.428308 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:24.428316 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:24.429774 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:24.431870 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:24.431877 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:24.431879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:24.432101 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:24.432346 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:24.432347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:24.432347 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:24.432468 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:24.432852 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:24.432858 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:24.432986 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 03:35:24.433088 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 03:35:24.433225 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260809 03:35:24.433398 18005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.423870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:43821 (local address 127.27.40.190:37221)
0809 03:35:24.423925 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:24.423927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:24.424082 (+ 155us) server_negotiation.cc:407] Connection header received
0809 03:35:24.424575 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:24.424576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:24.424588 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:24.424658 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:24.425720 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.426355 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.427400 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.427618 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.427684 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.429862 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:24.429863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:24.429864 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:24.429871 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:24.431785 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:24.432181 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:24.432183 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:24.432183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:24.432251 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:24.432548 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:24.432550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:24.432550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:24.432732 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:24.432807 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:24.433135 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 03:35:24.433239 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 03:35:24.434300 18432 heartbeater.cc:347] Connected to a master server at 127.27.40.190:37221
I20260809 03:35:24.434429 18432 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:24.434692 18432 heartbeater.cc:511] Master 127.27.40.190:37221 requested a full tablet report, sending...
I20260809 03:35:24.435195 17948 ts_manager.cc:194] Registered new tserver with Master: d959522beac347448ef72ef2860cc31a (127.27.40.130:40565)
I20260809 03:35:24.435595 17948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:43821
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:35:24.460939 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37221
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:34683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:24.544903 18437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:24.545163 18437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:24.545192 18437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:24.546520 18437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:24.546568 18437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:24.546599 18437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:35:24.548075 18437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:34683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:24.548321 18437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:24.548552 18437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:24.553874 18448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:24.553851 18447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:24.553841 18450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:24.554229 18437 server_base.cc:1034] running on GCE node
I20260809 03:35:24.554468 18437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:24.554704 18437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:24.555862 18437 hybrid_clock.cc:648] HybridClock initialized: now 1786246524555838 us; error 36 us; skew 500 ppm
Aug 09 03:35:24 dist-test-slave-gkw7 krb5kdc[17888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246524, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:24.558559 18437 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:35:24.559024 18437 webserver.cc:460] Webserver started at http://127.27.40.131:33733/ using document root <none> and password file <none>
I20260809 03:35:24.559269 18437 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:24.559337 18437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:24.559497 18437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:24.560516 18437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "b67787337a8347b49baf81d4aeed3d99"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "245e13e0aa90aaa8df91a308d5bd9767"
server_key_iv: "1fb25903912cdf62a71bbe0f8f709436"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.560882 18437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "b67787337a8347b49baf81d4aeed3d99"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "245e13e0aa90aaa8df91a308d5bd9767"
server_key_iv: "1fb25903912cdf62a71bbe0f8f709436"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.562353 18437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:35:24.563103 18463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:24.563282 18437 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:24.563377 18437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "b67787337a8347b49baf81d4aeed3d99"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "245e13e0aa90aaa8df91a308d5bd9767"
server_key_iv: "1fb25903912cdf62a71bbe0f8f709436"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.563453 18437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:24.577248 18437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:24.579145 18437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:24.579258 18437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:24.579422 18437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:24.579723 18437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:24.579756 18437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:24.579813 18437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:24.579844 18437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:24.585640 18437 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:42601
I20260809 03:35:24.585716 18632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:42601 every 8 connection(s)
I20260809 03:35:24.586056 18437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:24.586663 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 18437
I20260809 03:35:24.586740 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:35:24.586856 27810 external_mini_cluster.cc:1428] Setting key 0e7439ca80ba8082f5bb8922ff97bd4d
Aug 09 03:35:24 dist-test-slave-gkw7 krb5kdc[17888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246524, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:24.595494 18005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.587203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:40209 (local address 127.27.40.190:37221)
0809 03:35:24.587253 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:24.587256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:24.587573 (+ 317us) server_negotiation.cc:407] Connection header received
0809 03:35:24.588162 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:24.588163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:24.588176 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:24.588222 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:24.589141 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.589736 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.590746 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.590959 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.591046 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.593043 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:24.593044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:24.593045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:24.593051 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:24.594354 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:24.594717 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:24.594718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:24.594719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:24.594770 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:24.594973 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:24.594973 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:24.594974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:24.595072 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:24.595122 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:24.595360 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 03:35:24.595430 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 03:35:24.595645 18636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.587212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:40209)
0809 03:35:24.587433 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:24.587482 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:24.588062 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:24.588288 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:24.588292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:24.588857 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:24.589042 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.589047 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.589836 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.589837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.590637 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.590640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.591056 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.591057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.591191 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.591616 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:24.591623 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:24.592962 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:24.594424 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:24.594430 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:24.594432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:24.594653 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:24.594827 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:24.594828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:24.594828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:24.594921 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:24.595127 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:24.595145 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:24.595252 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 03:35:24.595330 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 03:35:24.595445 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":116,"threads_started":1}
I20260809 03:35:24.596124 18633 heartbeater.cc:347] Connected to a master server at 127.27.40.190:37221
I20260809 03:35:24.596238 18633 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:24.596412 18633 heartbeater.cc:511] Master 127.27.40.190:37221 requested a full tablet report, sending...
I20260809 03:35:24.596791 17948 ts_manager.cc:194] Registered new tserver with Master: b67787337a8347b49baf81d4aeed3d99 (127.27.40.131:42601)
I20260809 03:35:24.597169 17948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:40209
I20260809 03:35:24.598589 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:35:24.606009 18005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.600032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.27.40.190:37221)
0809 03:35:24.600067 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:24.600069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:24.600221 (+ 152us) server_negotiation.cc:407] Connection header received
0809 03:35:24.600286 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:24.600287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:24.600298 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:24.600365 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:24.600927 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.601391 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.602175 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.602427 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.602502 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.603670 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:24.603671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:24.603671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:24.603678 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:24.604795 (+ 1117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:24.605137 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:24.605138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:24.605138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:24.605195 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:24.605408 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:24.605408 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:24.605409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:24.605613 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:24.605728 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:24.605833 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 03:35:24.605912 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:35:24.607366 17948 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:35:24.608841 17948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:24.615595 18654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.611701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36272 (local address 127.27.40.131:42601)
0809 03:35:24.611851 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:24.611854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:24.611862 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:35:24.611890 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:24.611892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:24.611906 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:24.612033 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:24.612272 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.613314 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.614668 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.615175 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.615264 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.615325 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:24.615479 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 03:35:24.615520 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260809 03:35:24.615582 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.611240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:42601 (local address 127.0.0.1:36272)
0809 03:35:24.611636 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:24.611645 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:24.611684 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:24.612138 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:24.612139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:24.612140 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:24.612188 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.612190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.613424 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.613425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.614587 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.614589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.615266 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.615267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.615349 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.615361 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 03:35:24.615451 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 03:35:24.615505 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":35,"thread_start_us":52,"threads_started":1}
I20260809 03:35:24.615778 18651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.611250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:40565 (local address 127.0.0.1:37658)
0809 03:35:24.611870 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:24.611881 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:24.611947 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:24.612175 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:24.612176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:24.612178 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:24.612236 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.612238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.613121 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.613122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.614416 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.614419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.615413 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.615415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.615484 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.615488 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:35:24.615536 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 03:35:24.615578 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":99,"threads_started":1}
I20260809 03:35:24.615775 18653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.611529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37658 (local address 127.27.40.130:40565)
0809 03:35:24.611753 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:24.611757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:24.611961 (+ 204us) server_negotiation.cc:407] Connection header received
0809 03:35:24.612006 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:24.612008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:24.612024 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:24.612097 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:24.612322 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.613018 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.614498 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.615249 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.615321 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.615375 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:24.615636 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 03:35:24.615687 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":96,"threads_started":1}
I20260809 03:35:24.615959 18652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.611512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55638 (local address 127.27.40.129:34973)
0809 03:35:24.611677 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:24.611681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:24.611693 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:35:24.611736 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:24.611738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:24.611757 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:24.611851 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:24.612117 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.613317 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.614709 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.615423 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.615484 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.615532 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:24.615824 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 03:35:24.615879 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260809 03:35:24.615785 18648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.611238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:34973 (local address 127.0.0.1:55638)
0809 03:35:24.611397 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:24.611425 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:24.611517 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:24.611946 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:24.611950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:24.611951 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:24.612018 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.612020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.613435 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.613436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.614599 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.614602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.615565 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.615565 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.615619 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.615622 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:35:24.615671 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 03:35:24.615700 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":48,"spinlock_wait_cycles":384,"thread_start_us":104,"threads_started":1}
I20260809 03:35:24.616566 18331 tablet_service.cc:1467] Processing CreateTablet for tablet eabf633dfb0c47de97ac1f792d156b19 (DEFAULT_TABLE table=test-table [id=c3c7edfb4c254dbe990bd4596f39180f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:24.616830 18123 tablet_service.cc:1467] Processing CreateTablet for tablet eabf633dfb0c47de97ac1f792d156b19 (DEFAULT_TABLE table=test-table [id=c3c7edfb4c254dbe990bd4596f39180f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:24.616989 18331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eabf633dfb0c47de97ac1f792d156b19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:24.617182 18123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eabf633dfb0c47de97ac1f792d156b19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:24.617121 18531 tablet_service.cc:1467] Processing CreateTablet for tablet eabf633dfb0c47de97ac1f792d156b19 (DEFAULT_TABLE table=test-table [id=c3c7edfb4c254dbe990bd4596f39180f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:24.617408 18531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eabf633dfb0c47de97ac1f792d156b19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:24.619659 18655 tablet_bootstrap.cc:492] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b: Bootstrap starting.
I20260809 03:35:24.620388 18655 tablet_bootstrap.cc:654] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:24.620391 18656 tablet_bootstrap.cc:492] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a: Bootstrap starting.
I20260809 03:35:24.620625 18657 tablet_bootstrap.cc:492] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: Bootstrap starting.
I20260809 03:35:24.621079 18656 tablet_bootstrap.cc:654] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:24.621229 18655 log.cc:824] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:24.621403 18657 tablet_bootstrap.cc:654] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:24.621697 18656 log.cc:824] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:24.622033 18657 log.cc:824] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:24.622152 18655 tablet_bootstrap.cc:492] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b: No bootstrap required, opened a new log
I20260809 03:35:24.622229 18655 ts_tablet_manager.cc:1397] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:24.622503 18656 tablet_bootstrap.cc:492] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a: No bootstrap required, opened a new log
I20260809 03:35:24.622565 18656 ts_tablet_manager.cc:1397] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:35:24.622816 18657 tablet_bootstrap.cc:492] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: No bootstrap required, opened a new log
I20260809 03:35:24.622874 18657 ts_tablet_manager.cc:1397] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 03:35:24.623802 18656 raft_consensus.cc:348] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } }
I20260809 03:35:24.623922 18656 raft_consensus.cc:374] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:24.623942 18656 raft_consensus.cc:729] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d959522beac347448ef72ef2860cc31a, State: Initialized, Role: FOLLOWER
I20260809 03:35:24.624064 18656 consensus_queue.cc:260] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } }
I20260809 03:35:24.624352 18656 ts_tablet_manager.cc:1428] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:35:24.624249 18655 raft_consensus.cc:348] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } }
I20260809 03:35:24.624423 18655 raft_consensus.cc:374] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:24.624459 18655 raft_consensus.cc:729] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 328a09530d3d46db8d76405f32d1580b, State: Initialized, Role: FOLLOWER
I20260809 03:35:24.624478 18432 heartbeater.cc:503] Master 127.27.40.190:37221 was elected leader, sending a full tablet report...
I20260809 03:35:24.624570 18655 consensus_queue.cc:260] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } }
I20260809 03:35:24.624854 18655 ts_tablet_manager.cc:1428] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:24.624809 18657 raft_consensus.cc:348] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } }
I20260809 03:35:24.624920 18227 heartbeater.cc:503] Master 127.27.40.190:37221 was elected leader, sending a full tablet report...
I20260809 03:35:24.624969 18657 raft_consensus.cc:374] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:24.625012 18657 raft_consensus.cc:729] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b67787337a8347b49baf81d4aeed3d99, State: Initialized, Role: FOLLOWER
I20260809 03:35:24.625269 18657 consensus_queue.cc:260] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } }
I20260809 03:35:24.625589 18657 ts_tablet_manager.cc:1428] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 03:35:24.625699 18633 heartbeater.cc:503] Master 127.27.40.190:37221 was elected leader, sending a full tablet report...
I20260809 03:35:24.633481 18667 raft_consensus.cc:482] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:24.633636 18667 raft_consensus.cc:504] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } }
I20260809 03:35:24.634054 18667 leader_election.cc:288] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 328a09530d3d46db8d76405f32d1580b (127.27.40.129:34973), d959522beac347448ef72ef2860cc31a (127.27.40.130:40565)
I20260809 03:35:24.637890 18653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.634334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:36669 (local address 127.27.40.130:40565)
0809 03:35:24.634375 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:24.634377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:24.634384 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:35:24.634406 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:24.634407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:24.634421 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:24.634492 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:24.634678 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.635448 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.636954 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.637526 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.637638 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.637647 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:24.637790 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 03:35:24.637825 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 03:35:24.637890 18652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.634262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:42209 (local address 127.27.40.129:34973)
0809 03:35:24.634310 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:24.634312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:24.634472 (+ 160us) server_negotiation.cc:407] Connection header received
0809 03:35:24.634530 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:24.634530 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:24.634539 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:24.634593 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:24.634705 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.635448 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.636954 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.637565 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.637638 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.637647 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:24.637789 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 03:35:24.637825 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 03:35:24.637979 18636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.634190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:40565 (local address 127.27.40.131:36669)
0809 03:35:24.634244 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:24.634255 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:24.634296 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:24.634513 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:24.634515 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:24.634516 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:24.634588 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.634590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.635532 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.635533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.636855 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.636858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.637636 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.637637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.637722 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.637724 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 03:35:24.637780 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 03:35:24.637816 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":79}
I20260809 03:35:24.637925 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.634207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:34973 (local address 127.27.40.131:42209)
0809 03:35:24.634388 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:24.634398 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:24.634459 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:24.634588 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:24.634588 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:24.634590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:24.634651 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.634653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.635532 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.635533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.636855 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.636858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.637636 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.637637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.637715 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.637719 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:35:24.637774 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 03:35:24.637816 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":87,"thread_start_us":104,"threads_started":1}
I20260809 03:35:24.638299 18157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eabf633dfb0c47de97ac1f792d156b19" candidate_uuid: "b67787337a8347b49baf81d4aeed3d99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "328a09530d3d46db8d76405f32d1580b" is_pre_election: true
I20260809 03:35:24.638345 18362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eabf633dfb0c47de97ac1f792d156b19" candidate_uuid: "b67787337a8347b49baf81d4aeed3d99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d959522beac347448ef72ef2860cc31a" is_pre_election: true
I20260809 03:35:24.638446 18157 raft_consensus.cc:2454] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b67787337a8347b49baf81d4aeed3d99 in term 0.
I20260809 03:35:24.638468 18362 raft_consensus.cc:2454] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b67787337a8347b49baf81d4aeed3d99 in term 0.
I20260809 03:35:24.638690 18465 leader_election.cc:302] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 328a09530d3d46db8d76405f32d1580b, b67787337a8347b49baf81d4aeed3d99; no voters:
I20260809 03:35:24.638860 18667 raft_consensus.cc:2781] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:24.638916 18667 raft_consensus.cc:482] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:24.638950 18667 raft_consensus.cc:3037] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:24.639621 18667 raft_consensus.cc:504] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } }
I20260809 03:35:24.639770 18667 leader_election.cc:288] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [CANDIDATE]: Term 1 election: Requested vote from peers 328a09530d3d46db8d76405f32d1580b (127.27.40.129:34973), d959522beac347448ef72ef2860cc31a (127.27.40.130:40565)
I20260809 03:35:24.639933 18362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eabf633dfb0c47de97ac1f792d156b19" candidate_uuid: "b67787337a8347b49baf81d4aeed3d99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d959522beac347448ef72ef2860cc31a"
I20260809 03:35:24.639986 18157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eabf633dfb0c47de97ac1f792d156b19" candidate_uuid: "b67787337a8347b49baf81d4aeed3d99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "328a09530d3d46db8d76405f32d1580b"
I20260809 03:35:24.640013 18362 raft_consensus.cc:3037] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:24.640058 18157 raft_consensus.cc:3037] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:24.640712 18362 raft_consensus.cc:2454] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b67787337a8347b49baf81d4aeed3d99 in term 1.
I20260809 03:35:24.640796 18157 raft_consensus.cc:2454] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b67787337a8347b49baf81d4aeed3d99 in term 1.
I20260809 03:35:24.640925 18466 leader_election.cc:302] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b67787337a8347b49baf81d4aeed3d99, d959522beac347448ef72ef2860cc31a; no voters:
I20260809 03:35:24.641007 18667 raft_consensus.cc:2781] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:24.641152 18667 raft_consensus.cc:686] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 1 LEADER]: Becoming Leader. State: Replica: b67787337a8347b49baf81d4aeed3d99, State: Running, Role: LEADER
I20260809 03:35:24.641255 18667 consensus_queue.cc:237] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } }
I20260809 03:35:24.642125 17948 catalog_manager.cc:5495] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 reported cstate change: term changed from 0 to 1, leader changed from <none> to b67787337a8347b49baf81d4aeed3d99 (127.27.40.131). New cstate: current_term: 1 leader_uuid: "b67787337a8347b49baf81d4aeed3d99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } health_report { overall_health: UNKNOWN } } }
I20260809 03:35:24.652225 18654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.649818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36274 (local address 127.27.40.131:42601)
0809 03:35:24.649860 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:24.649862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:24.649872 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:35:24.649940 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:24.649941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:24.649958 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:24.650016 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:24.650176 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.650637 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.651409 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.651621 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.651693 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.651878 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:24.651983 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:24.652076 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:24.652111 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 03:35:24.652131 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 03:35:24.653795 18157 raft_consensus.cc:1261] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Refusing update from remote peer b67787337a8347b49baf81d4aeed3d99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:24.653816 18362 raft_consensus.cc:1261] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a [term 1 FOLLOWER]: Refusing update from remote peer b67787337a8347b49baf81d4aeed3d99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:24.654109 18667 consensus_queue.cc:1035] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:24.654271 18669 consensus_queue.cc:1035] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:24.656406 18674 mvcc.cc:205] Tried to move back new op lower bound from 7316465764979638272 to 7316465764931219456. Current Snapshot: MvccSnapshot[applied={T|T < 7316465764979638272}]
I20260809 03:35:24.656770 18675 mvcc.cc:205] Tried to move back new op lower bound from 7316465764979638272 to 7316465764931219456. Current Snapshot: MvccSnapshot[applied={T|T < 7316465764979638272}]
I20260809 03:35:24.657711 18676 mvcc.cc:205] Tried to move back new op lower bound from 7316465764979638272 to 7316465764931219456. Current Snapshot: MvccSnapshot[applied={T|T < 7316465764979638272 or (T in {7316465764979638272})}]
I20260809 03:35:24.659124 17947 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:24.659231 17947 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:24.660017 17947 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Sending DeleteTablet for 3 replicas of tablet eabf633dfb0c47de97ac1f792d156b19
I20260809 03:35:24.660364 18331 tablet_service.cc:1514] Processing DeleteTablet for tablet eabf633dfb0c47de97ac1f792d156b19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:24 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:37658
I20260809 03:35:24.660401 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 18040
I20260809 03:35:24.660415 18123 tablet_service.cc:1514] Processing DeleteTablet for tablet eabf633dfb0c47de97ac1f792d156b19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:24 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:55638
I20260809 03:35:24.669801 18531 tablet_service.cc:1514] Processing DeleteTablet for tablet eabf633dfb0c47de97ac1f792d156b19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:24 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:36272
I20260809 03:35:24.669950 18685 tablet_replica.cc:330] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a: stopping tablet replica
I20260809 03:35:24.670114 18685 raft_consensus.cc:2229] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:24.670305 18685 raft_consensus.cc:2258] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:24.670811 18685 ts_tablet_manager.cc:1905] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:24.672379 18685 ts_tablet_manager.cc:1918] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:24.672477 18685 log.cc:1196] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/eabf633dfb0c47de97ac1f792d156b19
I20260809 03:35:24.672806 18685 ts_tablet_manager.cc:1939] T eabf633dfb0c47de97ac1f792d156b19 P d959522beac347448ef72ef2860cc31a: Deleting consensus metadata
W20260809 03:35:24.673236 17928 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.40.129:34973 (error 108)
I20260809 03:35:24.673434 17929 catalog_manager.cc:4841] TS d959522beac347448ef72ef2860cc31a (127.27.40.130:40565): tablet eabf633dfb0c47de97ac1f792d156b19 (table test-table [id=c3c7edfb4c254dbe990bd4596f39180f]) successfully deleted
W20260809 03:35:24.673792 18651 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:24.673667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:34973 (local address 127.0.0.1:55652)
0809 03:35:24.673706 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:24.673718 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:24.673758 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:34973: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 03:35:24.673923 17928 catalog_manager.cc:4584] TS 328a09530d3d46db8d76405f32d1580b (127.27.40.129:34973): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eabf633dfb0c47de97ac1f792d156b19: Network error: Client connection negotiation failed: client connection to 127.27.40.129:34973: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260809 03:35:24.674268 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 18237
I20260809 03:35:24.685818 18686 tablet_replica.cc:330] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: stopping tablet replica
I20260809 03:35:24.686005 18686 raft_consensus.cc:2229] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:35:24.686106 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 18437
I20260809 03:35:24.686198 18686 raft_consensus.cc:2258] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:24.696856 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 17906
I20260809 03:35:24.696938 18651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.696750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:42601 (local address 127.0.0.1:36276)
0809 03:35:24.696819 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:24.696882 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.131:42601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 03:35:44.706734 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:37221
--webserver_interface=127.27.40.190
--webserver_port=37593
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:34683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:37221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:44.790086 18689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:44.790324 18689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:44.790361 18689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:44.791499 18689 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 03:35:44.791568 18689 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 03:35:44.791590 18689 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 03:35:44.791608 18689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:44.791636 18689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:44.791667 18689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:44.791692 18689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:44.791719 18689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:44.793146 18689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:34683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:37221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:37221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=37593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:44.793422 18689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:44.793673 18689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:44.803409 18699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:44.803442 18700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:44.803653 18689 server_base.cc:1034] running on GCE node
W20260809 03:35:44.803409 18702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:44.804003 18689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:44.804337 18689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:44.805513 18689 hybrid_clock.cc:648] HybridClock initialized: now 1786246544805500 us; error 46 us; skew 500 ppm
Aug 09 03:35:44 dist-test-slave-gkw7 krb5kdc[17888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246544, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:44.808250 18689 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:35:44.808749 18689 webserver.cc:460] Webserver started at http://127.27.40.190:37593/ using document root <none> and password file <none>
I20260809 03:35:44.808955 18689 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:44.809008 18689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:44.810527 18689 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:44.811203 18713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:44.811407 18689 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:44.811486 18689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "9a93316b6a59451db26e3fde483c11ea"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "6b83e9db128fb8140311fd6b722dc023"
server_key_iv: "b25c7bcf60d961d1837410695f05b603"
server_key_version: "encryptionkey@0"
I20260809 03:35:44.811913 18689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:44.821880 18689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:44.824049 18689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:44.824193 18689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:44.828400 18689 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:37221
I20260809 03:35:44.828454 18783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:37221 every 8 connection(s)
I20260809 03:35:44.828866 18689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:44.830472 18786 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:35:44.831187 18786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Bootstrap starting.
I20260809 03:35:44.833045 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 18689
I20260809 03:35:44.833422 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:34973
--local_ip_for_outbound_sockets=127.27.40.129
--tserver_master_addrs=127.27.40.190:37221
--webserver_port=41367
--webserver_interface=127.27.40.129
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:34683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:35:44.840574 18786 log.cc:824] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:44.842727 18786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:35:44.843048 18786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Bootstrap complete.
I20260809 03:35:44.845387 18786 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } }
I20260809 03:35:44.845674 18786 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a93316b6a59451db26e3fde483c11ea, State: Initialized, Role: FOLLOWER
I20260809 03:35:44.845858 18786 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } }
I20260809 03:35:44.845929 18786 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:44.846014 18786 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:44.846084 18786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:35:44.847102 18786 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } }
I20260809 03:35:44.847221 18786 leader_election.cc:302] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a93316b6a59451db26e3fde483c11ea; no voters:
I20260809 03:35:44.847409 18786 leader_election.cc:288] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 03:35:44.847468 18792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 2 FOLLOWER]: Leader election won for term 2
I20260809 03:35:44.847610 18792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 2 LEADER]: Becoming Leader. State: Replica: 9a93316b6a59451db26e3fde483c11ea, State: Running, Role: LEADER
I20260809 03:35:44.847682 18786 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:44.847703 18792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } }
I20260809 03:35:44.847945 18793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9a93316b6a59451db26e3fde483c11ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } } }
I20260809 03:35:44.848011 18793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:44.848140 18793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a93316b6a59451db26e3fde483c11ea. Latest consensus state: current_term: 2 leader_uuid: "9a93316b6a59451db26e3fde483c11ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } } }
I20260809 03:35:44.848187 18793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:44.848294 18799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:44.849072 18799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:44.849175 18799 catalog_manager.cc:1259] Loaded cluster ID: 509f493c6db64e56a5166d4a2268684a
I20260809 03:35:44.849207 18799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:44.850698 18799 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:44.850881 18799 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Loaded TSK: 0
I20260809 03:35:44.864296 18799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Generated new TSK 1
I20260809 03:35:44.864393 18799 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 03:35:44.923128 18790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:44.923323 18790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:44.923344 18790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:44.924463 18790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:44.924503 18790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:44.924530 18790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:35:44.926023 18790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:34683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:34973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=41367
--tserver_master_addrs=127.27.40.190:37221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:44.926247 18790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:44.926488 18790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:44.931414 18830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:44.931401 18829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:44.931703 18790 server_base.cc:1034] running on GCE node
W20260809 03:35:44.931485 18833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:44.932034 18790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:44.932291 18790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:44.933480 18790 hybrid_clock.cc:648] HybridClock initialized: now 1786246544933457 us; error 37 us; skew 500 ppm
Aug 09 03:35:44 dist-test-slave-gkw7 krb5kdc[17888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246544, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:44.936307 18790 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:35:44.936800 18790 webserver.cc:460] Webserver started at http://127.27.40.129:41367/ using document root <none> and password file <none>
I20260809 03:35:44.937095 18790 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:44.937206 18790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:44.938866 18790 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:44.939700 18844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:44.939893 18790 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:44.940009 18790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "328a09530d3d46db8d76405f32d1580b"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "211b307adc91b9bcb641cb3d9a47632c"
server_key_iv: "398f2f86002aa916f01815a981975f33"
server_key_version: "encryptionkey@0"
I20260809 03:35:44.940409 18790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:44.955204 18790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:44.957522 18790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:44.957780 18790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:44.958058 18790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:44.958541 18854 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 03:35:44.960409 18790 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 03:35:44.960486 18790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 03:35:44.960538 18790 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 03:35:44.961266 18790 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 03:35:44.961331 18790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 03:35:44.961382 18854 tablet_bootstrap.cc:492] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b: Bootstrap starting.
I20260809 03:35:44.969660 18790 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:34973
I20260809 03:35:44.969686 19028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:34973 every 8 connection(s)
I20260809 03:35:44.970131 18790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:44.972872 18854 log.cc:824] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:44.975625 18854 tablet_bootstrap.cc:492] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:35:44.975965 18854 tablet_bootstrap.cc:492] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b: Bootstrap complete.
I20260809 03:35:44.977128 18854 ts_tablet_manager.cc:1397] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.005s
I20260809 03:35:44.978663 18854 raft_consensus.cc:348] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } }
I20260809 03:35:44.978993 18854 raft_consensus.cc:729] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 328a09530d3d46db8d76405f32d1580b, State: Initialized, Role: FOLLOWER
I20260809 03:35:44.979166 18854 consensus_queue.cc:260] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } }
I20260809 03:35:44.979519 18854 ts_tablet_manager.cc:1428] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b: Time spent starting tablet: real 0.002s user 0.003s sys 0.003s
I20260809 03:35:44.979883 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 18790
I20260809 03:35:44.980199 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:40565
--local_ip_for_outbound_sockets=127.27.40.130
--tserver_master_addrs=127.27.40.190:37221
--webserver_port=39503
--webserver_interface=127.27.40.130
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:34683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:35:44 dist-test-slave-gkw7 krb5kdc[17888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246544, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:44.985296 19039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:44.971087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:56009 (local address 127.27.40.190:37221)
0809 03:35:44.973165 (+ 2078us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:44.973173 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:44.973207 (+ 34us) server_negotiation.cc:407] Connection header received
0809 03:35:44.973322 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:44.973327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:44.973373 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:44.973596 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:44.974475 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.975998 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.977663 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.978335 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.978466 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:44.981345 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:44.981349 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:44.981350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:44.981364 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:44.983333 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:44.983843 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:44.983846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:44.983847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:44.983950 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:44.984244 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:44.984246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:44.984246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:44.984494 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:44.984609 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:44.984935 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 03:35:44.985078 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2009,"thread_start_us":91,"threads_started":1}
I20260809 03:35:44.985297 19037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:44.970960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:56009)
0809 03:35:44.971151 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:44.971208 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:44.971713 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:44.973635 (+ 1922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:44.973639 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:44.974001 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:44.974301 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:44.974311 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.976211 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.976212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:44.977569 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:44.977573 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.978442 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.978443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:44.978577 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:44.979237 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:44.979247 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:44.981220 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:44.983455 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:44.983464 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:44.983466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:44.983727 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:44.984036 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:44.984037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:44.984037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:44.984163 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:44.984584 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:44.984592 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:44.984747 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 03:35:44.984873 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 03:35:44.985082 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260809 03:35:44.986191 19029 heartbeater.cc:347] Connected to a master server at 127.27.40.190:37221
I20260809 03:35:44.986292 19029 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:44.986491 19029 heartbeater.cc:511] Master 127.27.40.190:37221 requested a full tablet report, sending...
I20260809 03:35:44.987394 18733 ts_manager.cc:194] Registered new tserver with Master: 328a09530d3d46db8d76405f32d1580b (127.27.40.129:34973)
I20260809 03:35:44.988731 18733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:56009
I20260809 03:35:44.989759 19029 heartbeater.cc:503] Master 127.27.40.190:37221 was elected leader, sending a full tablet report...
I20260809 03:35:44.993716 19048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:44.988181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59208 (local address 127.27.40.129:34973)
0809 03:35:44.989958 (+ 1777us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:44.989963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:44.989973 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:35:44.990010 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:44.990012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:44.990029 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:44.990103 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:44.990378 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.991089 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.992750 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.993278 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.993345 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:44.993382 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:44.993616 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 03:35:44.993664 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1741,"thread_start_us":78,"threads_started":1}
I20260809 03:35:44.993737 19045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:44.988130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:34973 (local address 127.0.0.1:59208)
0809 03:35:44.989658 (+ 1528us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:44.989736 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:44.989811 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:44.990176 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:44.990179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:44.990182 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:44.990255 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:44.990257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.991186 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.991187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:44.992645 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:44.992649 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.993372 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.993373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:44.993464 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:44.993478 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 03:35:44.993572 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 03:35:44.993637 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1492,"thread_start_us":100,"threads_started":1}
I20260809 03:35:44.994282 18928 tablet_service.cc:1514] Processing DeleteTablet for tablet eabf633dfb0c47de97ac1f792d156b19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:24 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:59208
I20260809 03:35:44.994836 19050 tablet_replica.cc:330] stopping tablet replica
I20260809 03:35:44.994975 19050 raft_consensus.cc:2229] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:44.995075 19050 raft_consensus.cc:2258] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:44.995442 19050 ts_tablet_manager.cc:1905] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:44.997669 19050 ts_tablet_manager.cc:1918] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:44.997764 19050 log.cc:1196] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal/wals/eabf633dfb0c47de97ac1f792d156b19
I20260809 03:35:44.998009 19050 ts_tablet_manager.cc:1939] T eabf633dfb0c47de97ac1f792d156b19 P 328a09530d3d46db8d76405f32d1580b: Deleting consensus metadata
I20260809 03:35:44.998539 18715 catalog_manager.cc:4841] TS 328a09530d3d46db8d76405f32d1580b (127.27.40.129:34973): tablet eabf633dfb0c47de97ac1f792d156b19 (table test-table [id=c3c7edfb4c254dbe990bd4596f39180f]) successfully deleted
W20260809 03:35:45.070622 19043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:45.070943 19043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:45.070982 19043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:45.073032 19043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:45.073086 19043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:45.073130 19043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:35:45.075115 19043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:34683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:40565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=39503
--tserver_master_addrs=127.27.40.190:37221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:45.075404 19043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:45.075668 19043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:45.080885 19058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:45.080892 19060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:45.080904 19056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:45.081110 19043 server_base.cc:1034] running on GCE node
I20260809 03:35:45.081378 19043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:45.081666 19043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:45.082841 19043 hybrid_clock.cc:648] HybridClock initialized: now 1786246545082835 us; error 35 us; skew 500 ppm
Aug 09 03:35:45 dist-test-slave-gkw7 krb5kdc[17888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246545, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:45.085505 19043 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:35:45.086020 19043 webserver.cc:460] Webserver started at http://127.27.40.130:39503/ using document root <none> and password file <none>
I20260809 03:35:45.086268 19043 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:45.086333 19043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:45.087599 19043 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:45.088174 19067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:45.088377 19043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:45.088449 19043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "d959522beac347448ef72ef2860cc31a"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "4c231cad4f3cbb4a31c98ca1775563fe"
server_key_iv: "e36fef61f68f8eab5998f9c514196fb6"
server_key_version: "encryptionkey@0"
I20260809 03:35:45.088763 19043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:45.099679 19043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:45.101440 19043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:45.101608 19043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:45.101840 19043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:45.102077 19043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:45.102106 19043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:45.102128 19043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:45.102140 19043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:45.108057 19043 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:40565
I20260809 03:35:45.108162 19233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:40565 every 8 connection(s)
I20260809 03:35:45.108485 19043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
Aug 09 03:35:45 dist-test-slave-gkw7 krb5kdc[17888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246545, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:45.117131 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 19043
I20260809 03:35:45.117465 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:42601
--local_ip_for_outbound_sockets=127.27.40.131
--tserver_master_addrs=127.27.40.190:37221
--webserver_port=33733
--webserver_interface=127.27.40.131
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:34683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:35:45.118605 19039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.109396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:60333 (local address 127.27.40.190:37221)
0809 03:35:45.109454 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:45.109456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:45.109785 (+ 329us) server_negotiation.cc:407] Connection header received
0809 03:35:45.110258 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:45.110260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:45.110284 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:45.110361 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:45.111254 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.111830 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.112790 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.113006 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.113087 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.115465 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:45.115467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:45.115467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:45.115476 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:45.117007 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:45.117494 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:45.117495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:45.117496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:45.117591 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:45.117906 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:45.117908 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:45.117908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:45.118035 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:45.118095 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:45.118460 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 03:35:45.118540 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 03:35:45.118896 19237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.109392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:60333)
0809 03:35:45.109633 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:45.109677 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:45.110156 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:45.110412 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:45.110415 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:45.110978 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:45.111165 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.111170 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.111919 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.111920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.112694 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.112696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.113155 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.113156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.113271 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.113943 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:45.113951 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:45.115364 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:45.117119 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:45.117126 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:45.117128 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:45.117417 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:45.117684 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:45.117686 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:45.117687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:45.117832 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:45.118129 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:45.118136 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:45.118317 (+ 181us) client_negotiation.cc:769] Sending connection context
0809 03:35:45.118439 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 03:35:45.118620 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":107,"threads_started":1}
I20260809 03:35:45.119451 19235 heartbeater.cc:347] Connected to a master server at 127.27.40.190:37221
I20260809 03:35:45.119573 19235 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:45.119765 19235 heartbeater.cc:511] Master 127.27.40.190:37221 requested a full tablet report, sending...
I20260809 03:35:45.120195 18733 ts_manager.cc:194] Registered new tserver with Master: d959522beac347448ef72ef2860cc31a (127.27.40.130:40565)
I20260809 03:35:45.120600 18733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:60333
W20260809 03:35:45.200532 19238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:45.200736 19238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:45.200762 19238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:45.201925 19238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:45.201967 19238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:45.201993 19238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:35:45.203440 19238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:34683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:42601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=33733
--tserver_master_addrs=127.27.40.190:37221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:35:45.203671 19238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:45.203910 19238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:45.209055 19250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:45.209074 19245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:45.209132 19248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:45.209344 19238 server_base.cc:1034] running on GCE node
I20260809 03:35:45.209619 19238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:45.209887 19238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:45.211043 19238 hybrid_clock.cc:648] HybridClock initialized: now 1786246545211021 us; error 41 us; skew 500 ppm
Aug 09 03:35:45 dist-test-slave-gkw7 krb5kdc[17888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246545, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:45.213768 19238 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:35:45.214175 19238 webserver.cc:460] Webserver started at http://127.27.40.131:33733/ using document root <none> and password file <none>
I20260809 03:35:45.214360 19238 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:45.214409 19238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:45.215852 19238 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:45.216518 19257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:45.216696 19238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:45.216787 19238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "b67787337a8347b49baf81d4aeed3d99"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "245e13e0aa90aaa8df91a308d5bd9767"
server_key_iv: "1fb25903912cdf62a71bbe0f8f709436"
server_key_version: "encryptionkey@0"
I20260809 03:35:45.217120 19238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:45.264053 19238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:45.266261 19238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:45.266378 19238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:45.266582 19238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:45.267102 19266 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 03:35:45.268589 19238 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 03:35:45.268638 19238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 03:35:45.268664 19238 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 03:35:45.269333 19238 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 03:35:45.269366 19238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 03:35:45.269464 19266 tablet_bootstrap.cc:492] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: Bootstrap starting.
I20260809 03:35:45.276690 19238 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:42601
I20260809 03:35:45.276754 19429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:42601 every 8 connection(s)
I20260809 03:35:45.277182 19238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:45.280797 19266 log.cc:824] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:45.282992 19266 tablet_bootstrap.cc:492] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:35:45.283289 19266 tablet_bootstrap.cc:492] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: Bootstrap complete.
I20260809 03:35:45.284317 19266 ts_tablet_manager.cc:1397] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.006s
I20260809 03:35:45.284354 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 19238
I20260809 03:35:45.285676 19266 raft_consensus.cc:348] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } }
I20260809 03:35:45.285935 19266 raft_consensus.cc:729] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b67787337a8347b49baf81d4aeed3d99, State: Initialized, Role: FOLLOWER
Aug 09 03:35:45 dist-test-slave-gkw7 krb5kdc[17888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246545, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:35:45.286091 19266 consensus_queue.cc:260] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } }
I20260809 03:35:45.286375 19266 ts_tablet_manager.cc:1428] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260809 03:35:45.289511 19039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.278610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:45761 (local address 127.27.40.190:37221)
0809 03:35:45.278691 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:45.278693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:45.280359 (+ 1666us) server_negotiation.cc:407] Connection header received
0809 03:35:45.281052 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:45.281054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:45.281068 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:45.281139 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:45.281874 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.282393 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.283554 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.283773 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.283863 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.286624 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:45.286625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:45.286625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:45.286633 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:45.288107 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:45.288554 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:45.288555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:45.288556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:45.288619 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:45.288860 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:45.288861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:45.288861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:45.288976 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:45.289048 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:45.289324 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 03:35:45.289411 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:35:45.289680 19436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.278524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:45761)
0809 03:35:45.280188 (+ 1664us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:45.280240 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:45.280917 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:45.281198 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:45.281202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:45.281500 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:45.281761 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.281766 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.282503 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.282504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.283468 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.283470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.283872 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.283873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.284013 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.284600 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:45.284620 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:45.286482 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:45.288186 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:45.288192 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:45.288195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:45.288461 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:45.288687 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:45.288688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:45.288689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:45.288792 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:45.289050 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:45.289057 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:45.289188 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 03:35:45.289289 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 03:35:45.289445 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1613,"thread_start_us":105,"threads_started":1}
I20260809 03:35:45.290202 19431 heartbeater.cc:347] Connected to a master server at 127.27.40.190:37221
I20260809 03:35:45.290293 19431 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:45.290432 19431 heartbeater.cc:511] Master 127.27.40.190:37221 requested a full tablet report, sending...
I20260809 03:35:45.291014 18733 ts_manager.cc:194] Registered new tserver with Master: b67787337a8347b49baf81d4aeed3d99 (127.27.40.131:42601)
I20260809 03:35:45.291639 18733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:45761
I20260809 03:35:45.292501 19431 heartbeater.cc:503] Master 127.27.40.190:37221 was elected leader, sending a full tablet report...
I20260809 03:35:45.292739 19440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.284836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38584 (local address 127.27.40.190:37221)
0809 03:35:45.284952 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:45.284954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:45.285136 (+ 182us) server_negotiation.cc:407] Connection header received
0809 03:35:45.285196 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:45.285197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:45.285212 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:45.285288 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:45.286098 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.286781 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.287922 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.288135 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.288231 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.289419 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:45.289420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:45.289421 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:45.289428 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:45.291097 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:45.291536 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:45.291537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:45.291537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:45.291595 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:45.291811 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:45.291812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:45.291812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:45.291939 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:45.292079 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:45.292548 (+ 469us) server_negotiation.cc:299] Negotiation successful
0809 03:35:45.292630 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":66,"threads_started":1}
I20260809 03:35:45.293628 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 03:35:45 dist-test-slave-gkw7 krb5kdc[17888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246544, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for kudu/127.27.40.131@KRBTEST.COM
I20260809 03:35:45.296981 19440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.293968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38594 (local address 127.27.40.190:37221)
0809 03:35:45.294028 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:45.294029 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:45.294537 (+ 508us) server_negotiation.cc:407] Connection header received
0809 03:35:45.294636 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:45.294637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:45.294678 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:45.294776 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:45.294886 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.295314 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.296106 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.296437 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.296515 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.296705 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:45.296754 (+ 49us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 03:35:45.296918 (+ 164us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38594: token expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 03:35:45.297065 19440 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38594: token expired
W20260809 03:35:45.297211 18637 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.27.40.190:37221)
I20260809 03:35:45.299568 19045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.291323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:42601 (local address 127.0.0.1:47210)
0809 03:35:45.291372 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:45.291382 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:45.291440 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:45.291689 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:45.291690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:45.292056 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:45.292087 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.292089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.293164 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.293165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.293912 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.293915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.294360 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.294362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.294424 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.294904 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:45.294909 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:45.296259 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:45.298330 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:45.298336 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:45.298338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:45.298583 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:45.298847 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:45.298847 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:45.298848 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:45.298912 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:45.299220 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:45.299227 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:45.299277 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 03:35:45.299377 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 03:35:45.299478 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 03:35:45.299580 19444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.291386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47210 (local address 127.27.40.131:42601)
0809 03:35:45.291559 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:45.291563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:45.291570 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:35:45.291596 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:45.291597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:45.291608 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:45.291688 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:45.292175 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.293054 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.293979 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.294280 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.294361 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.296359 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:45.296365 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:45.296365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:45.296372 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:45.298255 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:45.298686 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:45.298689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:45.298689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:45.298766 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:45.298977 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:45.298978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:45.298978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:45.299117 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:45.299227 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:45.299411 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 03:35:45.299498 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":74,"threads_started":1}
I20260809 03:35:45.299955 19319 tablet_service.cc:1514] Processing DeleteTablet for tablet eabf633dfb0c47de97ac1f792d156b19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:24 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:47210
I20260809 03:35:45.300427 19449 tablet_replica.cc:330] stopping tablet replica
I20260809 03:35:45.300542 19449 raft_consensus.cc:2229] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:45.300608 19449 raft_consensus.cc:2258] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:45.300952 19449 ts_tablet_manager.cc:1905] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:45.303304 19449 ts_tablet_manager.cc:1918] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:45.303393 19449 log.cc:1196] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/eabf633dfb0c47de97ac1f792d156b19
I20260809 03:35:45.303615 19449 ts_tablet_manager.cc:1939] T eabf633dfb0c47de97ac1f792d156b19 P b67787337a8347b49baf81d4aeed3d99: Deleting consensus metadata
I20260809 03:35:45.303973 19440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.297467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38606 (local address 127.27.40.190:37221)
0809 03:35:45.297507 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:45.297509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:45.297657 (+ 148us) server_negotiation.cc:407] Connection header received
0809 03:35:45.297711 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:45.297712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:45.297722 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:45.297776 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:45.298177 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.298757 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.299494 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.299821 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.299886 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.301244 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:45.301246 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:45.301246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:45.301252 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:45.302774 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:45.303171 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:45.303172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:45.303172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:45.303231 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:45.303478 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:45.303479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:45.303479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:45.303596 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:45.303710 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:45.303808 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 03:35:45.303881 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:35:45.304121 18714 catalog_manager.cc:4841] TS b67787337a8347b49baf81d4aeed3d99 (127.27.40.131:42601): tablet eabf633dfb0c47de97ac1f792d156b19 (table test-table [id=c3c7edfb4c254dbe990bd4596f39180f]) successfully deleted
I20260809 03:35:45.305051 18733 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38606:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:35:45.306632 18733 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:45.308560 18928 tablet_service.cc:1467] Processing CreateTablet for tablet ed645785a2c442778a80bf47aa97dad2 (DEFAULT_TABLE table=test-table [id=a0780fed83124cf7909e1d6d0abf11c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:45.308676 19319 tablet_service.cc:1467] Processing CreateTablet for tablet ed645785a2c442778a80bf47aa97dad2 (DEFAULT_TABLE table=test-table [id=a0780fed83124cf7909e1d6d0abf11c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:45.308748 18928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed645785a2c442778a80bf47aa97dad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:45.308795 19319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed645785a2c442778a80bf47aa97dad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:45.310184 19266 tablet_bootstrap.cc:492] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99: Bootstrap starting.
I20260809 03:35:45.310314 18854 tablet_bootstrap.cc:492] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b: Bootstrap starting.
I20260809 03:35:45.310489 19266 tablet_bootstrap.cc:654] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:45.310634 18854 tablet_bootstrap.cc:654] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:45.311067 19266 tablet_bootstrap.cc:492] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99: No bootstrap required, opened a new log
I20260809 03:35:45.311139 19266 ts_tablet_manager.cc:1397] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:45.311226 18854 tablet_bootstrap.cc:492] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b: No bootstrap required, opened a new log
I20260809 03:35:45.311278 18854 ts_tablet_manager.cc:1397] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 03:35:45.311388 18854 raft_consensus.cc:348] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } }
I20260809 03:35:45.311472 18854 raft_consensus.cc:374] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:45.311480 19266 raft_consensus.cc:348] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } }
I20260809 03:35:45.311527 18854 raft_consensus.cc:729] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 328a09530d3d46db8d76405f32d1580b, State: Initialized, Role: FOLLOWER
I20260809 03:35:45.311555 19266 raft_consensus.cc:374] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:45.311597 19266 raft_consensus.cc:729] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b67787337a8347b49baf81d4aeed3d99, State: Initialized, Role: FOLLOWER
I20260809 03:35:45.311581 18854 consensus_queue.cc:260] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } }
I20260809 03:35:45.311640 19266 consensus_queue.cc:260] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } }
I20260809 03:35:45.311702 18854 ts_tablet_manager.cc:1428] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:45.311717 19266 ts_tablet_manager.cc:1428] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 03:35:45.313180 19451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.308537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55086 (local address 127.27.40.130:40565)
0809 03:35:45.309672 (+ 1135us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:45.309676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:45.309685 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:35:45.309720 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:45.309722 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:45.309735 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:45.309828 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:45.309921 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.310943 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.312244 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.312918 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.312973 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.313011 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:45.313094 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 03:35:45.313121 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1078,"thread_start_us":77,"threads_started":1}
I20260809 03:35:45.313189 19045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.308335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:40565 (local address 127.0.0.1:55086)
0809 03:35:45.308367 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:45.308376 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:45.308406 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:45.309824 (+ 1418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:45.309824 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:45.309827 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:45.309872 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.309874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.311049 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.311050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.312120 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.312123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.312984 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.312984 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.313033 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.313036 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:35:45.313083 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 03:35:45.313130 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260809 03:35:45.313987 19137 tablet_service.cc:1467] Processing CreateTablet for tablet ed645785a2c442778a80bf47aa97dad2 (DEFAULT_TABLE table=test-table [id=a0780fed83124cf7909e1d6d0abf11c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:45.314293 19137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed645785a2c442778a80bf47aa97dad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:45.317214 19456 tablet_bootstrap.cc:492] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a: Bootstrap starting.
I20260809 03:35:45.317979 19456 tablet_bootstrap.cc:654] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:45.318491 19456 log.cc:824] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:45.319315 19456 tablet_bootstrap.cc:492] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a: No bootstrap required, opened a new log
I20260809 03:35:45.319383 19456 ts_tablet_manager.cc:1397] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:35:45.320941 19456 raft_consensus.cc:348] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } }
I20260809 03:35:45.321144 19456 raft_consensus.cc:374] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:45.321208 19456 raft_consensus.cc:729] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d959522beac347448ef72ef2860cc31a, State: Initialized, Role: FOLLOWER
I20260809 03:35:45.321362 19456 consensus_queue.cc:260] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } }
I20260809 03:35:45.321764 19456 ts_tablet_manager.cc:1428] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260809 03:35:45.321810 19235 heartbeater.cc:503] Master 127.27.40.190:37221 was elected leader, sending a full tablet report...
W20260809 03:35:45.359097 19236 tablet.cc:2369] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:35:45.471160 19035 tablet.cc:2369] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:35:45.485016 19460 raft_consensus.cc:482] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:45.485157 19460 raft_consensus.cc:504] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } }
I20260809 03:35:45.485605 19460 leader_election.cc:288] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 328a09530d3d46db8d76405f32d1580b (127.27.40.129:34973), b67787337a8347b49baf81d4aeed3d99 (127.27.40.131:42601)
I20260809 03:35:45.488703 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.485839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:34973 (local address 127.27.40.130:45647)
0809 03:35:45.486051 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:45.486061 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:45.486146 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:45.486291 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:45.486292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:45.486293 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:45.486374 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.486375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.487024 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.487025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.487799 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.487801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.488384 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.488385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.488462 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.488465 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:35:45.488542 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 03:35:45.488583 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":114,"threads_started":1}
I20260809 03:35:45.488677 19048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.485917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:45647 (local address 127.27.40.129:34973)
0809 03:35:45.485965 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:45.485968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:45.486129 (+ 161us) server_negotiation.cc:407] Connection header received
0809 03:35:45.486202 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:45.486203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:45.486213 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:45.486287 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:45.486452 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.486938 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.487912 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.488295 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.488383 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.488393 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:45.488547 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 03:35:45.488586 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 03:35:45.489013 19237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.485800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:42601 (local address 127.27.40.130:58319)
0809 03:35:45.485874 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:45.485889 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:45.485952 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:45.486272 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:45.486273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:45.486276 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:45.486371 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.486374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.487141 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.487142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.487916 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.487918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.488709 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.488710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.488813 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.488817 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:35:45.488886 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 03:35:45.488927 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 03:35:45.489050 19444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.485989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:58319 (local address 127.27.40.131:42601)
0809 03:35:45.486106 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:45.486108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:45.486117 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:35:45.486145 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:45.486146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:45.486172 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:45.486267 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:45.486474 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.487038 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.488054 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.488560 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.488708 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.488812 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:45.488915 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 03:35:45.488971 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 03:35:45.489214 18957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed645785a2c442778a80bf47aa97dad2" candidate_uuid: "d959522beac347448ef72ef2860cc31a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "328a09530d3d46db8d76405f32d1580b" is_pre_election: true
I20260809 03:35:45.489367 18957 raft_consensus.cc:2454] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d959522beac347448ef72ef2860cc31a in term 0.
I20260809 03:35:45.489363 19353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed645785a2c442778a80bf47aa97dad2" candidate_uuid: "d959522beac347448ef72ef2860cc31a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b67787337a8347b49baf81d4aeed3d99" is_pre_election: true
I20260809 03:35:45.489532 19353 raft_consensus.cc:2454] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d959522beac347448ef72ef2860cc31a in term 0.
I20260809 03:35:45.489660 19072 leader_election.cc:302] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 328a09530d3d46db8d76405f32d1580b, d959522beac347448ef72ef2860cc31a; no voters:
I20260809 03:35:45.489828 19460 raft_consensus.cc:2781] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:45.489892 19460 raft_consensus.cc:482] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:45.489921 19460 raft_consensus.cc:3037] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:45.490906 19460 raft_consensus.cc:504] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } }
I20260809 03:35:45.491034 19460 leader_election.cc:288] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [CANDIDATE]: Term 1 election: Requested vote from peers 328a09530d3d46db8d76405f32d1580b (127.27.40.129:34973), b67787337a8347b49baf81d4aeed3d99 (127.27.40.131:42601)
I20260809 03:35:45.491215 18957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed645785a2c442778a80bf47aa97dad2" candidate_uuid: "d959522beac347448ef72ef2860cc31a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "328a09530d3d46db8d76405f32d1580b"
I20260809 03:35:45.491204 19353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed645785a2c442778a80bf47aa97dad2" candidate_uuid: "d959522beac347448ef72ef2860cc31a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b67787337a8347b49baf81d4aeed3d99"
I20260809 03:35:45.491300 18957 raft_consensus.cc:3037] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:45.491317 19353 raft_consensus.cc:3037] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:45.492038 18957 raft_consensus.cc:2454] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d959522beac347448ef72ef2860cc31a in term 1.
I20260809 03:35:45.492255 19353 raft_consensus.cc:2454] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d959522beac347448ef72ef2860cc31a in term 1.
I20260809 03:35:45.492256 19072 leader_election.cc:302] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 328a09530d3d46db8d76405f32d1580b, d959522beac347448ef72ef2860cc31a; no voters:
I20260809 03:35:45.492369 19460 raft_consensus.cc:2781] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:45.492522 19460 raft_consensus.cc:686] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [term 1 LEADER]: Becoming Leader. State: Replica: d959522beac347448ef72ef2860cc31a, State: Running, Role: LEADER
I20260809 03:35:45.492636 19460 consensus_queue.cc:237] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } }
I20260809 03:35:45.493459 18731 catalog_manager.cc:5495] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a reported cstate change: term changed from 0 to 1, leader changed from <none> to d959522beac347448ef72ef2860cc31a (127.27.40.130). New cstate: current_term: 1 leader_uuid: "d959522beac347448ef72ef2860cc31a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } health_report { overall_health: UNKNOWN } } }
W20260809 03:35:45.527930 19432 tablet.cc:2369] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:35:45.540738 19451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.538241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55102 (local address 127.27.40.130:40565)
0809 03:35:45.538311 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:45.538314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:45.538321 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:35:45.538364 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:45.538365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:45.538393 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:45.538516 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:45.538655 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.539134 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.539864 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.540093 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.540174 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.540351 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:45.540464 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:45.540558 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:45.540574 (+ 16us) server_negotiation.cc:299] Negotiation successful
0809 03:35:45.540594 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 03:35:45.542882 19353 raft_consensus.cc:1261] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99 [term 1 FOLLOWER]: Refusing update from remote peer d959522beac347448ef72ef2860cc31a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:45.542874 18957 raft_consensus.cc:1261] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Refusing update from remote peer d959522beac347448ef72ef2860cc31a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:45.543272 19463 consensus_queue.cc:1035] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [LEADER]: Connected to new peer: Peer: permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:45.543424 19460 consensus_queue.cc:1035] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:45.546515 19467 mvcc.cc:205] Tried to move back new op lower bound from 7316465850540363776 to 7316465850338463744. Current Snapshot: MvccSnapshot[applied={T|T < 7316465850540363776}]
I20260809 03:35:45.548151 18733 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:45.548254 18733 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:45.548990 18733 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Sending DeleteTablet for 3 replicas of tablet ed645785a2c442778a80bf47aa97dad2
I20260809 03:35:45.549321 18928 tablet_service.cc:1514] Processing DeleteTablet for tablet ed645785a2c442778a80bf47aa97dad2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:45 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:59208
I20260809 03:35:45.549342 19319 tablet_service.cc:1514] Processing DeleteTablet for tablet ed645785a2c442778a80bf47aa97dad2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:45 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:47210
I20260809 03:35:45.549422 19449 tablet_replica.cc:330] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99: stopping tablet replica
I20260809 03:35:45.549477 19449 raft_consensus.cc:2229] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:45.549569 19449 raft_consensus.cc:2258] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:45.549556 19137 tablet_service.cc:1514] Processing DeleteTablet for tablet ed645785a2c442778a80bf47aa97dad2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:45 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:55086
I20260809 03:35:45.549757 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 18790
I20260809 03:35:45.549777 19481 tablet_replica.cc:330] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a: stopping tablet replica
I20260809 03:35:45.549954 19481 raft_consensus.cc:2229] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:35:45.550108 19481 raft_consensus.cc:2258] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:45.552273 19481 ts_tablet_manager.cc:1905] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:45.554023 19481 ts_tablet_manager.cc:1918] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:45.554155 19481 log.cc:1196] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/ed645785a2c442778a80bf47aa97dad2
I20260809 03:35:45.554564 19481 ts_tablet_manager.cc:1939] T ed645785a2c442778a80bf47aa97dad2 P d959522beac347448ef72ef2860cc31a: Deleting consensus metadata
I20260809 03:35:45.555044 18717 catalog_manager.cc:4841] TS d959522beac347448ef72ef2860cc31a (127.27.40.130:40565): tablet ed645785a2c442778a80bf47aa97dad2 (table test-table [id=a0780fed83124cf7909e1d6d0abf11c9]) successfully deleted
I20260809 03:35:45.560046 19449 ts_tablet_manager.cc:1905] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:45.561580 19449 ts_tablet_manager.cc:1918] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:45.561647 19449 log.cc:1196] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/ed645785a2c442778a80bf47aa97dad2
I20260809 03:35:45.561882 19449 ts_tablet_manager.cc:1939] T ed645785a2c442778a80bf47aa97dad2 P b67787337a8347b49baf81d4aeed3d99: Deleting consensus metadata
I20260809 03:35:45.562294 18714 catalog_manager.cc:4841] TS b67787337a8347b49baf81d4aeed3d99 (127.27.40.131:42601): tablet ed645785a2c442778a80bf47aa97dad2 (table test-table [id=a0780fed83124cf7909e1d6d0abf11c9]) successfully deleted
W20260809 03:35:45.566386 18715 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.40.129:34973 (error 108)
I20260809 03:35:45.566931 19045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.566807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:34973 (local address 127.0.0.1:59212)
0809 03:35:45.566859 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:45.566896 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:34973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 03:35:45.566931 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 19043
W20260809 03:35:45.567044 18715 catalog_manager.cc:4584] TS 328a09530d3d46db8d76405f32d1580b (127.27.40.129:34973): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ed645785a2c442778a80bf47aa97dad2: Network error: Client connection negotiation failed: client connection to 127.27.40.129:34973: connect: Connection refused (error 111)
I20260809 03:35:45.577808 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 19238
I20260809 03:35:45.584964 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 18689
I20260809 03:36:05.596549 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:37221
--webserver_interface=127.27.40.190
--webserver_port=37593
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:34683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:37221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:05.684278 19483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:05.684495 19483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:05.684518 19483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:05.685778 19483 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 03:36:05.685819 19483 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 03:36:05.685832 19483 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 03:36:05.685844 19483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:05.685879 19483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:05.685909 19483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:05.685925 19483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:05.685948 19483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:05.687533 19483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:34683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:37221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:37221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=37593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:05.687805 19483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:05.688058 19483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:05.693493 19495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:05.693504 19492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:05.693586 19483 server_base.cc:1034] running on GCE node
W20260809 03:36:05.693504 19491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:05.693987 19483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:05.694310 19483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:05.695513 19483 hybrid_clock.cc:648] HybridClock initialized: now 1786246565695511 us; error 39 us; skew 500 ppm
Aug 09 03:36:05 dist-test-slave-gkw7 krb5kdc[17888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246565, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:05.698251 19483 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:36:05.698807 19483 webserver.cc:460] Webserver started at http://127.27.40.190:37593/ using document root <none> and password file <none>
I20260809 03:36:05.699090 19483 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:05.699173 19483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:05.700838 19483 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 03:36:05.701754 19504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:05.701988 19483 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:05.702102 19483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "9a93316b6a59451db26e3fde483c11ea"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "6b83e9db128fb8140311fd6b722dc023"
server_key_iv: "b25c7bcf60d961d1837410695f05b603"
server_key_version: "encryptionkey@0"
I20260809 03:36:05.702473 19483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:05.722910 19483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:05.724923 19483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:05.725103 19483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:05.729326 19483 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:37221
I20260809 03:36:05.729368 19581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:37221 every 8 connection(s)
I20260809 03:36:05.729827 19483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:05.731376 19582 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:36:05.732069 19582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Bootstrap starting.
I20260809 03:36:05.733423 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 19483
I20260809 03:36:05.733776 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:34973
--local_ip_for_outbound_sockets=127.27.40.129
--tserver_master_addrs=127.27.40.190:37221
--webserver_port=41367
--webserver_interface=127.27.40.129
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:34683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:36:05.741267 19582 log.cc:824] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:05.743613 19582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:36:05.743964 19582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Bootstrap complete.
I20260809 03:36:05.746394 19582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } }
I20260809 03:36:05.746640 19582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a93316b6a59451db26e3fde483c11ea, State: Initialized, Role: FOLLOWER
I20260809 03:36:05.746852 19582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } }
I20260809 03:36:05.746951 19582 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:05.747005 19582 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:05.747053 19582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 2 FOLLOWER]: Advancing to term 3
I20260809 03:36:05.748044 19582 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } }
I20260809 03:36:05.748179 19582 leader_election.cc:302] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a93316b6a59451db26e3fde483c11ea; no voters:
I20260809 03:36:05.748379 19582 leader_election.cc:288] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 03:36:05.748433 19588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 3 FOLLOWER]: Leader election won for term 3
I20260809 03:36:05.748606 19588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [term 3 LEADER]: Becoming Leader. State: Replica: 9a93316b6a59451db26e3fde483c11ea, State: Running, Role: LEADER
I20260809 03:36:05.748675 19582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:05.748834 19588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } }
I20260809 03:36:05.749214 19588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a93316b6a59451db26e3fde483c11ea. Latest consensus state: current_term: 3 leader_uuid: "9a93316b6a59451db26e3fde483c11ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } } }
I20260809 03:36:05.749306 19590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9a93316b6a59451db26e3fde483c11ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a93316b6a59451db26e3fde483c11ea" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37221 } } }
I20260809 03:36:05.749387 19588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:05.749411 19590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:05.750053 19602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:05.750988 19602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:05.751092 19602 catalog_manager.cc:1259] Loaded cluster ID: 509f493c6db64e56a5166d4a2268684a
I20260809 03:36:05.751124 19602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:05.752400 19602 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:05.752583 19602 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Loaded TSK: 0
I20260809 03:36:05.752627 19602 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9a93316b6a59451db26e3fde483c11ea: Loaded TSK: 1
F20260809 03:36:05.752864 19602 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f38cd1229ad google::LogMessage::Fail() at ??:0
@ 0x7f38cd12588a google::LogMessage::SendToLog() at ??:0
@ 0x7f38cd1224de google::LogMessage::Flush() at ??:0
@ 0x7f38cd123a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f38cf88f104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f38cf88cd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f38d507c1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f38d507c48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f38d507c594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f38d5042c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f38d504cad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f38ceff00d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f38cefe5954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f38cfe496db start_thread at ??:0
@ 0x7f38cbc2061f clone at ??:0
W20260809 03:36:05.838598 19586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:05.838814 19586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:05.838837 19586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:05.839987 19586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:05.840029 19586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:05.840054 19586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:36:05.841522 19586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:34683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:34973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=41367
--tserver_master_addrs=127.27.40.190:37221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:05.841876 19586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:05.842118 19586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:05.847349 19649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:05.847365 19643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:05.847465 19645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:05.847695 19586 server_base.cc:1034] running on GCE node
I20260809 03:36:05.847887 19586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:05.848140 19586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:05.849303 19586 hybrid_clock.cc:648] HybridClock initialized: now 1786246565849292 us; error 26 us; skew 500 ppm
Aug 09 03:36:05 dist-test-slave-gkw7 krb5kdc[17888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246565, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:05.852003 19586 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:36:05.852437 19586 webserver.cc:460] Webserver started at http://127.27.40.129:41367/ using document root <none> and password file <none>
I20260809 03:36:05.852689 19586 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:05.852761 19586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:05.854275 19586 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 03:36:05.855064 19659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:05.855293 19586 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:36:05.855417 19586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "328a09530d3d46db8d76405f32d1580b"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "211b307adc91b9bcb641cb3d9a47632c"
server_key_iv: "398f2f86002aa916f01815a981975f33"
server_key_version: "encryptionkey@0"
I20260809 03:36:05.855744 19586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:05.867509 19586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:05.869735 19586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:05.869896 19586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:05.870146 19586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:05.870663 19668 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 03:36:05.872723 19586 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 03:36:05.872768 19586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 03:36:05.872812 19586 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 03:36:05.873461 19586 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 03:36:05.873519 19586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 03:36:05.873627 19668 tablet_bootstrap.cc:492] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b: Bootstrap starting.
I20260809 03:36:05.881017 19586 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:34973
I20260809 03:36:05.881170 19829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:34973 every 8 connection(s)
I20260809 03:36:05.881474 19586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:36:05.884990 19668 log.cc:824] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:05.887931 19668 tablet_bootstrap.cc:492] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:36:05.888284 19668 tablet_bootstrap.cc:492] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b: Bootstrap complete.
I20260809 03:36:05.889608 19668 ts_tablet_manager.cc:1397] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.013s
I20260809 03:36:05.890252 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 19586
I20260809 03:36:05.890558 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:40565
--local_ip_for_outbound_sockets=127.27.40.130
--tserver_master_addrs=127.27.40.190:37221
--webserver_port=39503
--webserver_interface=127.27.40.130
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:34683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:36:05.891129 19668 raft_consensus.cc:348] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } }
I20260809 03:36:05.891511 19668 raft_consensus.cc:729] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 328a09530d3d46db8d76405f32d1580b, State: Initialized, Role: FOLLOWER
I20260809 03:36:05.891681 19668 consensus_queue.cc:260] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } }
I20260809 03:36:05.891955 19668 ts_tablet_manager.cc:1428] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260809 03:36:05.976158 19834 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:36:05.882183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:52129)
0809 03:36:05.882359 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:05.882403 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:05.882865 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:05.975919 (+ 93054us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":103,"threads_started":1}
W20260809 03:36:05.976442 19830 heartbeater.cc:644] Failed to heartbeat to 127.27.40.190:37221 (0 consecutive failures): Network error: Failed to ping master at 127.27.40.190:37221: Client connection negotiation failed: client connection to 127.27.40.190:37221: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 03:36:05.976862 19834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:05.976720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:42455)
0809 03:36:05.976785 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:05.976836 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 03:36:05.977228 19834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:05.977120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:37007)
0809 03:36:05.977178 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:05.977203 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 03:36:05.977365 19830 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:36:05.992875 19840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:05.993083 19840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:05.993108 19840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:05.994266 19840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:05.994309 19840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:05.994335 19840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:36:05.995771 19840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:34683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:40565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=39503
--tserver_master_addrs=127.27.40.190:37221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:05.995980 19840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:05.996157 19840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:06.001418 19848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:06.001554 19840 server_base.cc:1034] running on GCE node
W20260809 03:36:06.001368 19852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:06.001368 19849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:06.001902 19840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:06.002107 19840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:06.003283 19840 hybrid_clock.cc:648] HybridClock initialized: now 1786246566003271 us; error 28 us; skew 500 ppm
Aug 09 03:36:05 dist-test-slave-gkw7 krb5kdc[17888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246565, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:06.005789 19840 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:36:06.006256 19840 webserver.cc:460] Webserver started at http://127.27.40.130:39503/ using document root <none> and password file <none>
I20260809 03:36:06.006500 19840 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:06.006572 19840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:06.007953 19840 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:06.008589 19862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:06.008775 19840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:36:06.008878 19840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "d959522beac347448ef72ef2860cc31a"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "4c231cad4f3cbb4a31c98ca1775563fe"
server_key_iv: "e36fef61f68f8eab5998f9c514196fb6"
server_key_version: "encryptionkey@0"
I20260809 03:36:06.009256 19840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:06.029405 19840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:06.031448 19840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:06.031632 19840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:06.031895 19840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:06.032245 19840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:06.032310 19840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:06.032353 19840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:06.032382 19840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:06.039028 19840 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:40565
I20260809 03:36:06.039075 20034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:40565 every 8 connection(s)
I20260809 03:36:06.039494 19840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:36:06.040851 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:06.040351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:45157)
0809 03:36:06.040581 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:06.040667 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":93,"threads_started":1}
W20260809 03:36:06.041396 20037 heartbeater.cc:644] Failed to heartbeat to 127.27.40.190:37221 (0 consecutive failures): Network error: Failed to ping master at 127.27.40.190:37221: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
I20260809 03:36:06.041746 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:06.041643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:38981)
0809 03:36:06.041691 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:06.041724 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 03:36:06.042158 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:06.042043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:50943)
0809 03:36:06.042114 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:06.042141 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 03:36:06.042330 20037 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 03:36:06.047943 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 19840
I20260809 03:36:06.048305 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:42601
--local_ip_for_outbound_sockets=127.27.40.131
--tserver_master_addrs=127.27.40.190:37221
--webserver_port=33733
--webserver_interface=127.27.40.131
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:34683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:06.132242 20042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:06.132467 20042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:06.132489 20042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:06.133589 20042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:06.133627 20042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:06.133653 20042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:36:06.135068 20042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:34683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:42601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=33733
--tserver_master_addrs=127.27.40.190:37221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:06.135272 20042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:06.135458 20042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:06.140771 20051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:06.140793 20048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:06.140995 20049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:06.141284 20042 server_base.cc:1034] running on GCE node
I20260809 03:36:06.141562 20042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:06.141855 20042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:06.143031 20042 hybrid_clock.cc:648] HybridClock initialized: now 1786246566143003 us; error 42 us; skew 500 ppm
Aug 09 03:36:06 dist-test-slave-gkw7 krb5kdc[17888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246566, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:06.145649 20042 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:36:06.146047 20042 webserver.cc:460] Webserver started at http://127.27.40.131:33733/ using document root <none> and password file <none>
I20260809 03:36:06.146258 20042 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:06.146306 20042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:06.147591 20042 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:06.148180 20063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:06.148361 20042 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:36:06.148470 20042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "b67787337a8347b49baf81d4aeed3d99"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-gkw7"
server_key: "245e13e0aa90aaa8df91a308d5bd9767"
server_key_iv: "1fb25903912cdf62a71bbe0f8f709436"
server_key_version: "encryptionkey@0"
I20260809 03:36:06.148844 20042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:06.164829 20042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:06.167341 20042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:06.167515 20042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:06.167765 20042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:06.168083 20042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:06.168140 20042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:06.168186 20042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:06.168219 20042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:06.174685 20042 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:42601
I20260809 03:36:06.174750 20224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:42601 every 8 connection(s)
I20260809 03:36:06.175135 20042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:36:06.176265 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:06.175879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:33531)
0809 03:36:06.176040 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:06.176113 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
W20260809 03:36:06.176697 20227 heartbeater.cc:644] Failed to heartbeat to 127.27.40.190:37221 (0 consecutive failures): Network error: Failed to ping master at 127.27.40.190:37221: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
I20260809 03:36:06.176982 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:06.176892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:44461)
0809 03:36:06.176937 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:06.176965 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 03:36:06.177227 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:06.177168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:40037)
0809 03:36:06.177194 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:06.177213 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260809 03:36:06.177335 20227 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 03:36:06.184846 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 20042
I20260809 03:36:06.978040 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:06.977760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:58569)
0809 03:36:06.977902 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:06.977956 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
I20260809 03:36:07.042937 20233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:07.042677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:48897)
0809 03:36:07.042843 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:07.042881 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":69,"threads_started":1}
I20260809 03:36:07.177908 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:07.177668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:52977)
0809 03:36:07.177810 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:07.177849 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":56,"threads_started":1}
I20260809 03:36:07.545192 20235 raft_consensus.cc:482] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:36:07.545359 20235 raft_consensus.cc:504] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } }
I20260809 03:36:07.545796 20235 leader_election.cc:288] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d959522beac347448ef72ef2860cc31a (127.27.40.130:40565), b67787337a8347b49baf81d4aeed3d99 (127.27.40.131:42601)
Aug 09 03:36:07 dist-test-slave-gkw7 krb5kdc[17888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246565, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.131@KRBTEST.COM
Aug 09 03:36:07 dist-test-slave-gkw7 krb5kdc[17888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246565, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.130@KRBTEST.COM
I20260809 03:36:07.561422 20239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:07.546120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:37117 (local address 127.27.40.131:42601)
0809 03:36:07.546349 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:07.546358 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:07.546422 (+ 64us) server_negotiation.cc:407] Connection header received
0809 03:36:07.546863 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:07.546865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:07.546903 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:07.547055 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:07.547812 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:07.548731 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:07.549771 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:07.550163 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:07.550288 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:07.557307 (+ 7019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:07.557327 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:07.557328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:07.557343 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:07.559493 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:07.559948 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:07.559951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:07.559951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:07.560056 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:07.560327 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:07.560328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:07.560328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:07.560671 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:07.560766 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:07.561159 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 03:36:07.561303 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":64,"threads_started":1}
I20260809 03:36:07.561422 20238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:07.546060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:56493 (local address 127.27.40.130:40565)
0809 03:36:07.546316 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:07.546329 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:07.546357 (+ 28us) server_negotiation.cc:407] Connection header received
0809 03:36:07.546857 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:07.546860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:07.546903 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:07.547063 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:07.547907 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:07.548749 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:07.549757 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:07.550163 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:07.550283 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:07.557680 (+ 7397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:07.557685 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:07.557685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:07.557698 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:07.559745 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:07.560090 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:07.560093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:07.560093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:07.560204 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:07.560402 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:07.560403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:07.560403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:07.560686 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:07.560766 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:07.561159 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 03:36:07.561304 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":54,"threads_started":1}
I20260809 03:36:07.561419 20237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:07.545977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:42601 (local address 127.27.40.129:37117)
0809 03:36:07.546207 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:07.546217 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:07.546263 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:07.547086 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:07.547089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:07.547506 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:07.547690 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:07.547696 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:07.548839 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:07.548840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:07.549678 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:07.549680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:07.550263 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:07.550264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:07.550374 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:07.555473 (+ 5099us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:07.555482 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:07.557178 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:07.559579 (+ 2401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:07.559586 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:07.559587 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:07.559859 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:07.560129 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:07.560130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:07.560130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:07.560250 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:07.560761 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:07.560771 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:07.560941 (+ 170us) client_negotiation.cc:769] Sending connection context
0809 03:36:07.561082 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 03:36:07.561291 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":4,"thread_start_us":107,"threads_started":1}
I20260809 03:36:07.561467 20236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:07.545974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:40565 (local address 127.27.40.129:56493)
0809 03:36:07.546102 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:07.546114 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:07.546196 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:07.547087 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:07.547090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:07.547505 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:07.547689 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:07.547696 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:07.548852 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:07.548853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:07.549672 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:07.549675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:07.550267 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:07.550268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:07.550374 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:07.555501 (+ 5127us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:07.555506 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:07.557513 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:07.559823 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:07.559824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:07.559825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:07.560014 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:07.560271 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:07.560272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:07.560272 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:07.560331 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:07.560761 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:07.560771 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:07.560941 (+ 170us) client_negotiation.cc:769] Sending connection context
0809 03:36:07.561082 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 03:36:07.561348 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":172,"thread_start_us":57,"threads_started":1}
W20260809 03:36:07.562791 19660 leader_election.cc:341] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b67787337a8347b49baf81d4aeed3d99 (127.27.40.131:42601): Not found: Tablet not found: ed645785a2c442778a80bf47aa97dad2
W20260809 03:36:07.562903 19662 leader_election.cc:341] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d959522beac347448ef72ef2860cc31a (127.27.40.130:40565): Not found: Tablet not found: ed645785a2c442778a80bf47aa97dad2
I20260809 03:36:07.562951 19662 leader_election.cc:302] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 328a09530d3d46db8d76405f32d1580b; no voters: b67787337a8347b49baf81d4aeed3d99, d959522beac347448ef72ef2860cc31a
I20260809 03:36:07.563069 20235 raft_consensus.cc:2726] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:36:07.978744 20236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:07.978600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:34219)
0809 03:36:07.978655 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:07.978698 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 03:36:08.043726 20240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:08.043452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:37019)
0809 03:36:08.043603 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:08.043662 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260809 03:36:08.178623 20241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:08.178363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:58641)
0809 03:36:08.178511 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:08.178562 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":63,"threads_started":1}
I20260809 03:36:08.979453 20242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:08.979238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:38245)
0809 03:36:08.979367 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:08.979405 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260809 03:36:09.044498 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:09.044256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:45217)
0809 03:36:09.044411 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:09.044451 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":68,"threads_started":1}
I20260809 03:36:09.179373 20248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:09.179143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:54817)
0809 03:36:09.179284 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:09.179327 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
I20260809 03:36:09.326858 20249 raft_consensus.cc:482] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:36:09.326960 20249 raft_consensus.cc:504] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } }
I20260809 03:36:09.327163 20249 leader_election.cc:288] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d959522beac347448ef72ef2860cc31a (127.27.40.130:40565), b67787337a8347b49baf81d4aeed3d99 (127.27.40.131:42601)
W20260809 03:36:09.327601 19660 leader_election.cc:341] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b67787337a8347b49baf81d4aeed3d99 (127.27.40.131:42601): Not found: Tablet not found: ed645785a2c442778a80bf47aa97dad2
W20260809 03:36:09.327726 19662 leader_election.cc:341] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d959522beac347448ef72ef2860cc31a (127.27.40.130:40565): Not found: Tablet not found: ed645785a2c442778a80bf47aa97dad2
I20260809 03:36:09.327792 19662 leader_election.cc:302] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 328a09530d3d46db8d76405f32d1580b; no voters: b67787337a8347b49baf81d4aeed3d99, d959522beac347448ef72ef2860cc31a
I20260809 03:36:09.327876 20249 raft_consensus.cc:2726] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:36:09.980144 20250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:09.979916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:53905)
0809 03:36:09.980058 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:09.980096 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260809 03:36:10.045281 20251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:10.044993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:47633)
0809 03:36:10.045157 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:10.045199 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":61,"threads_started":1}
I20260809 03:36:10.180006 20252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:10.179796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:38759)
0809 03:36:10.179930 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:10.179968 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260809 03:36:10.980891 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:10.980668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:33319)
0809 03:36:10.980801 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:10.980843 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
I20260809 03:36:11.046132 20254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:11.045867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:42207)
0809 03:36:11.046020 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:11.046065 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":58,"threads_started":1}
I20260809 03:36:11.180757 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:11.180513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:51511)
0809 03:36:11.180671 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:11.180711 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260809 03:36:11.851689 20258 raft_consensus.cc:482] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:36:11.851799 20258 raft_consensus.cc:504] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } }
I20260809 03:36:11.851958 20258 leader_election.cc:288] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d959522beac347448ef72ef2860cc31a (127.27.40.130:40565), b67787337a8347b49baf81d4aeed3d99 (127.27.40.131:42601)
W20260809 03:36:11.852382 19662 leader_election.cc:341] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d959522beac347448ef72ef2860cc31a (127.27.40.130:40565): Not found: Tablet not found: ed645785a2c442778a80bf47aa97dad2
W20260809 03:36:11.852476 19660 leader_election.cc:341] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b67787337a8347b49baf81d4aeed3d99 (127.27.40.131:42601): Not found: Tablet not found: ed645785a2c442778a80bf47aa97dad2
I20260809 03:36:11.852619 19660 leader_election.cc:302] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 328a09530d3d46db8d76405f32d1580b; no voters: b67787337a8347b49baf81d4aeed3d99, d959522beac347448ef72ef2860cc31a
I20260809 03:36:11.852715 20258 raft_consensus.cc:2726] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:36:11.981730 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:11.981440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:58903)
0809 03:36:11.981634 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:11.981679 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":72,"threads_started":1}
I20260809 03:36:12.046907 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:12.046663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:43899)
0809 03:36:12.046813 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:12.046858 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260809 03:36:12.181519 20263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:12.181278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:47249)
0809 03:36:12.181425 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:12.181474 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
I20260809 03:36:12.982451 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:12.982210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:49181)
0809 03:36:12.982362 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:12.982403 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":65,"threads_started":1}
I20260809 03:36:13.047709 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:13.047429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:39593)
0809 03:36:13.047580 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:13.047621 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":64,"threads_started":1}
I20260809 03:36:13.182363 20267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:13.182112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:55929)
0809 03:36:13.182267 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:13.182311 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
I20260809 03:36:13.983238 20268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:13.982999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:45053)
0809 03:36:13.983134 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:13.983187 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
I20260809 03:36:14.048574 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:14.048288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:60385)
0809 03:36:14.048474 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:14.048526 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":107,"threads_started":1}
I20260809 03:36:14.183164 20271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:14.182930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:37905)
0809 03:36:14.183077 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:14.183115 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260809 03:36:14.984030 20274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:14.983774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:56429)
0809 03:36:14.983936 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:14.983981 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":70,"threads_started":1}
I20260809 03:36:15.049314 20275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:15.049093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:33963)
0809 03:36:15.049222 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:15.049266 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260809 03:36:15.183940 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:15.183717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:55121)
0809 03:36:15.183853 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:15.183895 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260809 03:36:15.624246 20277 raft_consensus.cc:482] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:36:15.624352 20277 raft_consensus.cc:504] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } }
I20260809 03:36:15.624588 20277 leader_election.cc:288] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d959522beac347448ef72ef2860cc31a (127.27.40.130:40565), b67787337a8347b49baf81d4aeed3d99 (127.27.40.131:42601)
W20260809 03:36:15.625047 19662 leader_election.cc:341] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d959522beac347448ef72ef2860cc31a (127.27.40.130:40565): Not found: Tablet not found: ed645785a2c442778a80bf47aa97dad2
W20260809 03:36:15.625129 19660 leader_election.cc:341] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b67787337a8347b49baf81d4aeed3d99 (127.27.40.131:42601): Not found: Tablet not found: ed645785a2c442778a80bf47aa97dad2
I20260809 03:36:15.625180 19660 leader_election.cc:302] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 328a09530d3d46db8d76405f32d1580b; no voters: b67787337a8347b49baf81d4aeed3d99, d959522beac347448ef72ef2860cc31a
I20260809 03:36:15.625267 20277 raft_consensus.cc:2726] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:36:15.984892 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:15.984627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:39247)
0809 03:36:15.984792 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:15.984841 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":72,"threads_started":1}
I20260809 03:36:16.050308 20281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:16.049941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:33819)
0809 03:36:16.050114 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:16.050164 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":83,"threads_started":1}
I20260809 03:36:16.184654 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:16.184409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:52897)
0809 03:36:16.184556 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:16.184595 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
I20260809 03:36:16.985706 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:16.985416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:42457)
0809 03:36:16.985618 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:16.985660 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":68,"threads_started":1}
I20260809 03:36:17.051151 20289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:17.050888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:33425)
0809 03:36:17.051033 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:17.051083 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260809 03:36:17.185364 20293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:17.185127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:46821)
0809 03:36:17.185285 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:17.185322 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260809 03:36:17.986469 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:17.986230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:48003)
0809 03:36:17.986375 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:17.986419 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
I20260809 03:36:18.052115 20297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:18.051865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:58115)
0809 03:36:18.052017 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:18.052065 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260809 03:36:18.186077 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:18.185836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:54045)
0809 03:36:18.185971 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:18.186016 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260809 03:36:18.987223 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:18.986975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:46755)
0809 03:36:18.987130 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:18.987174 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":64,"threads_started":1}
I20260809 03:36:19.052848 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:19.052640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:43737)
0809 03:36:19.052765 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:19.052802 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260809 03:36:19.186968 20302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:19.186676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:38899)
0809 03:36:19.186854 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:19.186904 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260809 03:36:19.988131 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:19.987871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:38345)
0809 03:36:19.988041 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:19.988082 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":84,"threads_started":1}
I20260809 03:36:20.053563 20305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:20.053276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:46669)
0809 03:36:20.053407 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:20.053467 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
I20260809 03:36:20.187650 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:20.187444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:58441)
0809 03:36:20.187567 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:20.187605 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260809 03:36:20.988965 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:20.988705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.129:45769)
0809 03:36:20.988877 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:20.988917 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":68,"threads_started":1}
I20260809 03:36:21.054350 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:21.054135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.130:56451)
0809 03:36:21.054268 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:21.054303 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260809 03:36:21.087893 20314 raft_consensus.cc:482] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:36:21.088001 20314 raft_consensus.cc:504] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d959522beac347448ef72ef2860cc31a" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 40565 } } peers { permanent_uuid: "328a09530d3d46db8d76405f32d1580b" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 34973 } } peers { permanent_uuid: "b67787337a8347b49baf81d4aeed3d99" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 42601 } }
I20260809 03:36:21.088203 20314 leader_election.cc:288] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d959522beac347448ef72ef2860cc31a (127.27.40.130:40565), b67787337a8347b49baf81d4aeed3d99 (127.27.40.131:42601)
W20260809 03:36:21.088598 19660 leader_election.cc:341] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b67787337a8347b49baf81d4aeed3d99 (127.27.40.131:42601): Not found: Tablet not found: ed645785a2c442778a80bf47aa97dad2
W20260809 03:36:21.088680 19662 leader_election.cc:341] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d959522beac347448ef72ef2860cc31a (127.27.40.130:40565): Not found: Tablet not found: ed645785a2c442778a80bf47aa97dad2
I20260809 03:36:21.088716 19662 leader_election.cc:302] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 328a09530d3d46db8d76405f32d1580b; no voters: b67787337a8347b49baf81d4aeed3d99, d959522beac347448ef72ef2860cc31a
I20260809 03:36:21.088789 20314 raft_consensus.cc:2726] T ed645785a2c442778a80bf47aa97dad2 P 328a09530d3d46db8d76405f32d1580b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:36:21.188481 20315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:21.188227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37221 (local address 127.27.40.131:56499)
0809 03:36:21.188387 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:21.188431 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.190:37221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":76,"threads_started":1}
I20260809 03:36:21.188692 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 19586
I20260809 03:36:21.200922 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 19840
I20260809 03:36:21.211300 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 20042
2026-08-09T03:36:21Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58711 ms)
[----------] 30 tests from SecurityITest (241530 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:21 dist-test-slave-gkw7 krb5kdc[20318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:21 dist-test-slave-gkw7 krb5kdc[20318](info): set up 2 sockets
Aug 09 03:36:21 dist-test-slave-gkw7 krb5kdc[20318](info): commencing operation
krb5kdc: starting...
W20260809 03:36:22.480331 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:22 dist-test-slave-gkw7 krb5kdc[20318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:36:22.612869 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:38813
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:38813
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:22.696337 20340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:22.696604 20340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:22.696645 20340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:22.697814 20340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:22.697886 20340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:22.697911 20340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:22.697932 20340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:22.697961 20340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:22.699440 20340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:38813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:38813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:22.699733 20340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:22.699965 20340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:22.705453 20354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:22.705470 20352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:22.705439 20350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:22.705472 20340 server_base.cc:1034] running on GCE node
I20260809 03:36:22.705942 20340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:22.706259 20340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:22.707402 20340 hybrid_clock.cc:648] HybridClock initialized: now 1786246582707391 us; error 31 us; skew 500 ppm
Aug 09 03:36:22 dist-test-slave-gkw7 krb5kdc[20318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246582, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:22.710152 20340 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:36:22.710623 20340 webserver.cc:460] Webserver started at http://127.27.40.190:45561/ using document root <none> and password file <none>
I20260809 03:36:22.710866 20340 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:22.710937 20340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:22.711091 20340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:22.712162 20340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "12b1c1b4e2a04750acb8ef4c98c9ddc7"
format_stamp: "Formatted at 2026-08-09 03:36:22 on dist-test-slave-gkw7"
server_key: "da46ce11214fd3f2e55689dcee1bfc92"
server_key_iv: "c49a4d8b00c6c9f877ea3ef91461f9e2"
server_key_version: "encryptionkey@0"
I20260809 03:36:22.712535 20340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "12b1c1b4e2a04750acb8ef4c98c9ddc7"
format_stamp: "Formatted at 2026-08-09 03:36:22 on dist-test-slave-gkw7"
server_key: "da46ce11214fd3f2e55689dcee1bfc92"
server_key_iv: "c49a4d8b00c6c9f877ea3ef91461f9e2"
server_key_version: "encryptionkey@0"
I20260809 03:36:22.713965 20340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:36:22.714718 20363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:22.714902 20340 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:36:22.715029 20340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "12b1c1b4e2a04750acb8ef4c98c9ddc7"
format_stamp: "Formatted at 2026-08-09 03:36:22 on dist-test-slave-gkw7"
server_key: "da46ce11214fd3f2e55689dcee1bfc92"
server_key_iv: "c49a4d8b00c6c9f877ea3ef91461f9e2"
server_key_version: "encryptionkey@0"
I20260809 03:36:22.715121 20340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:22.756012 20340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:22.758586 20340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:22.758769 20340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:22.763038 20340 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:38813
I20260809 03:36:22.763092 20438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:38813 every 8 connection(s)
I20260809 03:36:22.763494 20340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:22.764385 20440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:22.766726 20440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7: Bootstrap starting.
I20260809 03:36:22.767329 20440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:22.767637 20440 log.cc:824] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:22.768386 20440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7: No bootstrap required, opened a new log
I20260809 03:36:22.768747 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 20340
I20260809 03:36:22.768872 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:22.769063 27810 external_mini_cluster.cc:1428] Setting key f06ce43b0b65f9d8cf7ca3f6c431d6b8
I20260809 03:36:22.770385 20440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12b1c1b4e2a04750acb8ef4c98c9ddc7" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38813 } }
I20260809 03:36:22.770550 20440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:22.770596 20440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12b1c1b4e2a04750acb8ef4c98c9ddc7, State: Initialized, Role: FOLLOWER
I20260809 03:36:22.770804 20440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12b1c1b4e2a04750acb8ef4c98c9ddc7" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38813 } }
I20260809 03:36:22.770906 20440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:22.770947 20440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:22.770999 20440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:22.771867 20440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12b1c1b4e2a04750acb8ef4c98c9ddc7" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38813 } }
I20260809 03:36:22.771986 20440 leader_election.cc:302] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12b1c1b4e2a04750acb8ef4c98c9ddc7; no voters:
I20260809 03:36:22.772198 20440 leader_election.cc:288] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:22.772228 20449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:22.772336 20449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [term 1 LEADER]: Becoming Leader. State: Replica: 12b1c1b4e2a04750acb8ef4c98c9ddc7, State: Running, Role: LEADER
I20260809 03:36:22.772472 20440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:22.772483 20449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12b1c1b4e2a04750acb8ef4c98c9ddc7" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38813 } }
I20260809 03:36:22.772964 20451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12b1c1b4e2a04750acb8ef4c98c9ddc7. Latest consensus state: current_term: 1 leader_uuid: "12b1c1b4e2a04750acb8ef4c98c9ddc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12b1c1b4e2a04750acb8ef4c98c9ddc7" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38813 } } }
I20260809 03:36:22.773144 20451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:22.773689 20466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:22.774159 20466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:22.775274 20450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12b1c1b4e2a04750acb8ef4c98c9ddc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12b1c1b4e2a04750acb8ef4c98c9ddc7" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38813 } } }
I20260809 03:36:22.775358 20450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:22.776566 20466 catalog_manager.cc:1347] Generated new cluster ID: 3673101ccdd74e328d5b45181bdb86db
I20260809 03:36:22.776626 20466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 03:36:22 dist-test-slave-gkw7 krb5kdc[20318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:22.781895 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:22.769630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46082 (local address 127.27.40.190:38813)
0809 03:36:22.769844 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:22.769854 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:22.769889 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:36:22.770379 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:22.770385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:22.770435 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:22.770611 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:22.771423 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:22.772527 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:22.773643 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:22.774255 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:22.774360 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:22.777412 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:22.777416 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:22.777416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:22.777427 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:22.780013 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:22.780377 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:22.780380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:22.780380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:22.780491 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:22.780675 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:22.780675 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:22.780676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:22.780946 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:22.781076 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:22.781510 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 03:36:22.781674 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":80,"threads_started":1}
I20260809 03:36:22.789078 20466 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:22.789695 20466 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:22.794953 20466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7: Generated new TSK 0
I20260809 03:36:22.795141 20466 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:36:22.830411 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38813
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:22.914165 20494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:22.914369 20494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:22.914391 20494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:22.915515 20494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:22.915565 20494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:22.915591 20494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:36:22.916999 20494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:22.917237 20494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:22.917466 20494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:22.922720 20506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:22.922761 20501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:22.922719 20502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:22.923130 20494 server_base.cc:1034] running on GCE node
I20260809 03:36:22.923336 20494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:22.923593 20494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:22.924767 20494 hybrid_clock.cc:648] HybridClock initialized: now 1786246582924745 us; error 38 us; skew 500 ppm
Aug 09 03:36:22 dist-test-slave-gkw7 krb5kdc[20318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246582, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:22.927520 20494 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:36:22.928136 20494 webserver.cc:460] Webserver started at http://127.27.40.129:38599/ using document root <none> and password file <none>
I20260809 03:36:22.928367 20494 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:22.928438 20494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:22.928598 20494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:22.929733 20494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "e0b1d251e143403cae66634c4dc62b2d"
format_stamp: "Formatted at 2026-08-09 03:36:22 on dist-test-slave-gkw7"
server_key: "0e63254f26cd3e322b5116dabf788679"
server_key_iv: "5f8fd39bfe372264bb327a33622088cd"
server_key_version: "encryptionkey@0"
I20260809 03:36:22.930097 20494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "e0b1d251e143403cae66634c4dc62b2d"
format_stamp: "Formatted at 2026-08-09 03:36:22 on dist-test-slave-gkw7"
server_key: "0e63254f26cd3e322b5116dabf788679"
server_key_iv: "5f8fd39bfe372264bb327a33622088cd"
server_key_version: "encryptionkey@0"
I20260809 03:36:22.931449 20494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 03:36:22.932235 20514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:22.932430 20494 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:36:22.932545 20494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "e0b1d251e143403cae66634c4dc62b2d"
format_stamp: "Formatted at 2026-08-09 03:36:22 on dist-test-slave-gkw7"
server_key: "0e63254f26cd3e322b5116dabf788679"
server_key_iv: "5f8fd39bfe372264bb327a33622088cd"
server_key_version: "encryptionkey@0"
I20260809 03:36:22.932637 20494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:22.941823 20494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:22.943711 20494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:22.943873 20494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:22.944113 20494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:22.944422 20494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:22.944479 20494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:22.944519 20494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:22.944553 20494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:22.950930 20494 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:42471
I20260809 03:36:22.950987 20686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:42471 every 8 connection(s)
I20260809 03:36:22.951306 20494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:36:22.956282 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 20494
I20260809 03:36:22.956416 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:36:22.956611 27810 external_mini_cluster.cc:1428] Setting key 24490f650ce71418017b3cf09552ac53
Aug 09 03:36:22 dist-test-slave-gkw7 krb5kdc[20318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246582, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:22.962010 20692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:22.951964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:38813 (local address 127.27.40.129:35919)
0809 03:36:22.952152 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:22.952197 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:22.952673 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:22.952917 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:22.952921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:22.953594 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:22.953813 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:22.953820 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:22.954546 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:22.954547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:22.955370 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:22.955372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:22.955801 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:22.955802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:22.955908 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:22.956508 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:22.956519 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:22.958489 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:22.960445 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:22.960453 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:22.960455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:22.960690 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:22.960912 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:22.960912 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:22.960913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:22.961017 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:22.961348 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:22.961355 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:22.961507 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 03:36:22.961674 (+ 167us) client_negotiation.cc:240] Negotiation successful
0809 03:36:22.961828 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
I20260809 03:36:22.962000 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:22.952037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:35919 (local address 127.27.40.190:38813)
0809 03:36:22.952112 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:22.952114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:22.952295 (+ 181us) server_negotiation.cc:407] Connection header received
0809 03:36:22.952764 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:22.952765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:22.952776 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:22.952841 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:22.953927 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:22.954452 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:22.955484 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:22.955713 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:22.955781 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:22.958596 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:22.958598 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:22.958598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:22.958607 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:22.960351 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:22.960771 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:22.960773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:22.960773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:22.960840 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:22.961086 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:22.961087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:22.961087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:22.961242 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:22.961315 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:22.961714 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 03:36:22.961836 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:36:22.962944 20689 heartbeater.cc:347] Connected to a master server at 127.27.40.190:38813
I20260809 03:36:22.963066 20689 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:22.963263 20689 heartbeater.cc:511] Master 127.27.40.190:38813 requested a full tablet report, sending...
I20260809 03:36:22.963829 20385 ts_manager.cc:194] Registered new tserver with Master: e0b1d251e143403cae66634c4dc62b2d (127.27.40.129:42471)
I20260809 03:36:22.964465 20385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:35919
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:36:22.984995 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38813
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:23.068545 20695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:23.068776 20695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:23.068816 20695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:23.070071 20695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:23.070139 20695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:23.070186 20695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:36:23.071672 20695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:23.071951 20695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:23.072197 20695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:23.077677 20710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:23.077759 20711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:23.077659 20713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:23.077890 20695 server_base.cc:1034] running on GCE node
I20260809 03:36:23.078094 20695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:23.078382 20695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:23.079566 20695 hybrid_clock.cc:648] HybridClock initialized: now 1786246583079542 us; error 28 us; skew 500 ppm
Aug 09 03:36:23 dist-test-slave-gkw7 krb5kdc[20318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246583, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:23.082088 20695 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:36:23.082546 20695 webserver.cc:460] Webserver started at http://127.27.40.130:43963/ using document root <none> and password file <none>
I20260809 03:36:23.082806 20695 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:23.082880 20695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:23.083036 20695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:23.084089 20695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "db0768400157468b92f4c9032f9899ba"
format_stamp: "Formatted at 2026-08-09 03:36:23 on dist-test-slave-gkw7"
server_key: "7c2eb7c0abe98f440d74779eb75ec6ee"
server_key_iv: "4c4cb86a6a89ada8475142715abca245"
server_key_version: "encryptionkey@0"
I20260809 03:36:23.084502 20695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "db0768400157468b92f4c9032f9899ba"
format_stamp: "Formatted at 2026-08-09 03:36:23 on dist-test-slave-gkw7"
server_key: "7c2eb7c0abe98f440d74779eb75ec6ee"
server_key_iv: "4c4cb86a6a89ada8475142715abca245"
server_key_version: "encryptionkey@0"
I20260809 03:36:23.085917 20695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:36:23.086673 20728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:23.086846 20695 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:36:23.086954 20695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "db0768400157468b92f4c9032f9899ba"
format_stamp: "Formatted at 2026-08-09 03:36:23 on dist-test-slave-gkw7"
server_key: "7c2eb7c0abe98f440d74779eb75ec6ee"
server_key_iv: "4c4cb86a6a89ada8475142715abca245"
server_key_version: "encryptionkey@0"
I20260809 03:36:23.087029 20695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:23.107511 20695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:23.109441 20695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:23.109650 20695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:23.109931 20695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:23.110297 20695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:23.110356 20695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:23.110392 20695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:23.110419 20695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:23.116469 20695 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:45101
I20260809 03:36:23.116518 20892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:45101 every 8 connection(s)
I20260809 03:36:23.116854 20695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:36:23.120714 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 20695
I20260809 03:36:23.120827 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:36:23.121016 27810 external_mini_cluster.cc:1428] Setting key 56049dea81c3a56e275e5db49d74ecc4
Aug 09 03:36:23 dist-test-slave-gkw7 krb5kdc[20318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246583, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:23.127588 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.117640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:33739 (local address 127.27.40.190:38813)
0809 03:36:23.117715 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:23.117718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:23.117989 (+ 271us) server_negotiation.cc:407] Connection header received
0809 03:36:23.118449 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:23.118450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:23.118462 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:23.118518 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:23.119589 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.120102 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.121479 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.121830 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.121905 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.124660 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:23.124662 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:23.124662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:23.124669 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:23.126074 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:23.126609 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:23.126610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:23.126610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:23.126663 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:23.126955 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:23.126955 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:23.126955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:23.127054 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:23.127101 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:23.127450 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 03:36:23.127523 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 03:36:23.127890 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.117676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:38813 (local address 127.27.40.130:33739)
0809 03:36:23.117859 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:23.117896 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:23.118368 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:23.118597 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:23.118601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:23.119295 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:23.119495 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.119500 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.120183 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.120184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.121378 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.121381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.121925 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.121926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.122099 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.122638 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:23.122648 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:23.124563 (+ 1915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:23.126161 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:23.126168 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:23.126170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:23.126532 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:23.126730 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:23.126731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:23.126732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:23.126884 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:23.127118 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:23.127125 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:23.127301 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 03:36:23.127426 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 03:36:23.127625 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
I20260809 03:36:23.128432 20893 heartbeater.cc:347] Connected to a master server at 127.27.40.190:38813
I20260809 03:36:23.128554 20893 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:23.128746 20893 heartbeater.cc:511] Master 127.27.40.190:38813 requested a full tablet report, sending...
I20260809 03:36:23.129163 20385 ts_manager.cc:194] Registered new tserver with Master: db0768400157468b92f4c9032f9899ba (127.27.40.130:45101)
I20260809 03:36:23.129597 20385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:33739
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:36:23.152206 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38813
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:23.237432 20898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:23.237645 20898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:23.237669 20898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:23.238746 20898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:23.238780 20898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:23.238806 20898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:36:23.240191 20898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:23.240373 20898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:23.240538 20898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:36:23.245599 20898 server_base.cc:1034] running on GCE node
W20260809 03:36:23.245548 20913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:23.245572 20906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:23.245573 20909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:23.245980 20898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:23.246285 20898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:23.247452 20898 hybrid_clock.cc:648] HybridClock initialized: now 1786246583247427 us; error 38 us; skew 500 ppm
Aug 09 03:36:23 dist-test-slave-gkw7 krb5kdc[20318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246583, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:23.250252 20898 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:36:23.250720 20898 webserver.cc:460] Webserver started at http://127.27.40.131:41163/ using document root <none> and password file <none>
I20260809 03:36:23.250955 20898 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:23.251022 20898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:23.251181 20898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:23.252194 20898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "62c3debfae0e477e83c557fb65181bad"
format_stamp: "Formatted at 2026-08-09 03:36:23 on dist-test-slave-gkw7"
server_key: "1810c4afce640b41911b05aa8f394bfc"
server_key_iv: "0ce3ed82d1732bfa828f51ff6183afd6"
server_key_version: "encryptionkey@0"
I20260809 03:36:23.252555 20898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "62c3debfae0e477e83c557fb65181bad"
format_stamp: "Formatted at 2026-08-09 03:36:23 on dist-test-slave-gkw7"
server_key: "1810c4afce640b41911b05aa8f394bfc"
server_key_iv: "0ce3ed82d1732bfa828f51ff6183afd6"
server_key_version: "encryptionkey@0"
I20260809 03:36:23.253962 20898 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:36:23.254738 20929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:23.254922 20898 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:36:23.255026 20898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "62c3debfae0e477e83c557fb65181bad"
format_stamp: "Formatted at 2026-08-09 03:36:23 on dist-test-slave-gkw7"
server_key: "1810c4afce640b41911b05aa8f394bfc"
server_key_iv: "0ce3ed82d1732bfa828f51ff6183afd6"
server_key_version: "encryptionkey@0"
I20260809 03:36:23.255098 20898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:23.304522 20898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:23.307024 20898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:23.307200 20898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:23.307446 20898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:23.307788 20898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:23.307857 20898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:23.307916 20898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:23.307947 20898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:23.314167 20898 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:33111
I20260809 03:36:23.314224 21102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:33111 every 8 connection(s)
I20260809 03:36:23.314579 20898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:36:23.318364 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 20898
I20260809 03:36:23.318467 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:36:23.318739 27810 external_mini_cluster.cc:1428] Setting key 323aee85e44e216bbb312f80a51361d6
Aug 09 03:36:23 dist-test-slave-gkw7 krb5kdc[20318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246583, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:23.324860 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.315356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:45857 (local address 127.27.40.190:38813)
0809 03:36:23.315430 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:23.315432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:23.315605 (+ 173us) server_negotiation.cc:407] Connection header received
0809 03:36:23.316110 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:23.316111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:23.316123 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:23.316196 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:23.317152 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.317920 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.319518 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.319863 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.319952 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.322365 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:23.322366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:23.322367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:23.322375 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:23.323651 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:23.324048 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:23.324049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:23.324049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:23.324101 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:23.324354 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:23.324354 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:23.324354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:23.324452 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:23.324501 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:23.324735 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 03:36:23.324796 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 03:36:23.325146 21105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.315319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:38813 (local address 127.27.40.131:45857)
0809 03:36:23.315474 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:23.315515 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:23.315993 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:23.316258 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:23.316262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:23.316831 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:23.317029 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.317033 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.318040 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.318041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.319408 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.319411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.319958 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.319958 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.320066 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.320634 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:23.320643 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:23.322270 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:23.323737 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:23.323744 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:23.323745 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:23.323977 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:23.324165 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:23.324166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:23.324166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:23.324294 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:23.324508 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:23.324513 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:23.324625 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 03:36:23.324706 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 03:36:23.324824 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260809 03:36:23.325691 21103 heartbeater.cc:347] Connected to a master server at 127.27.40.190:38813
I20260809 03:36:23.325788 21103 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:23.325930 21103 heartbeater.cc:511] Master 127.27.40.190:38813 requested a full tablet report, sending...
I20260809 03:36:23.326273 20385 ts_manager.cc:194] Registered new tserver with Master: 62c3debfae0e477e83c557fb65181bad (127.27.40.131:33111)
I20260809 03:36:23.326666 20385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:45857
I20260809 03:36:23.330483 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:36:23.338109 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.331785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46096 (local address 127.27.40.190:38813)
0809 03:36:23.331853 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:23.331854 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:23.331912 (+ 58us) server_negotiation.cc:407] Connection header received
0809 03:36:23.331950 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:23.331951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:23.331959 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:23.332003 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:23.332563 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.333106 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.333855 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.334033 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.334113 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.335396 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:23.335398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:23.335398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:23.335404 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:23.336946 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:23.337305 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:23.337306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:23.337306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:23.337358 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:23.337612 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:23.337618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:23.337618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:23.337774 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:23.337868 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:23.337982 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 03:36:23.338041 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:36:23.339691 20385 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:36:23.341239 20385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:36:23.348083 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.343717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:45101 (local address 127.0.0.1:41362)
0809 03:36:23.343920 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:23.343964 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:23.344039 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:23.344416 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:23.344420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:23.344423 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:23.344511 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.344513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.345522 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.345523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.346807 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.346810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.347776 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.347777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.347864 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.347880 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 03:36:23.347950 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 03:36:23.348005 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":71,"threads_started":1}
I20260809 03:36:23.348135 21120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.343847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41362 (local address 127.27.40.130:45101)
0809 03:36:23.344199 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:23.344220 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:23.344232 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:36:23.344275 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:23.344278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:23.344294 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:23.344386 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:23.344595 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.345417 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.346906 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.347645 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.347770 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.347845 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:23.347964 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 03:36:23.348006 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":79,"threads_started":1}
I20260809 03:36:23.348341 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.343934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:33111 (local address 127.0.0.1:33704)
0809 03:36:23.344085 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:23.344096 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:23.344184 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:23.344556 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:23.344557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:23.344561 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:23.344628 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.344630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.346027 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.346028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.346982 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.346986 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.348072 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.348073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.348151 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.348155 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:36:23.348215 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 03:36:23.348270 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":60,"threads_started":1}
I20260809 03:36:23.348598 21124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.344117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33704 (local address 127.27.40.131:33111)
0809 03:36:23.344301 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:23.344305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:23.344314 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:36:23.344351 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:23.344353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:23.344368 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:23.344442 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:23.344778 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.345920 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.347125 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.347893 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.347971 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.348032 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:23.348441 (+ 409us) server_negotiation.cc:299] Negotiation successful
0809 03:36:23.348496 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260809 03:36:23.348960 21119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.343757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:42471 (local address 127.0.0.1:54238)
0809 03:36:23.344014 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:23.344024 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:23.344069 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:23.344558 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:23.344559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:23.344561 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:23.344596 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.344597 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.346003 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.346004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.347165 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.347167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.348588 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.348589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.348743 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.348747 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:36:23.348836 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 03:36:23.348876 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":24,"thread_start_us":92,"threads_started":1}
I20260809 03:36:23.348892 20796 tablet_service.cc:1467] Processing CreateTablet for tablet f8ff55e5a0af4dc19227aefe84c44779 (DEFAULT_TABLE table=test-table [id=d3a88d36ffad46d2bbea0b0113b805ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:23.349123 21121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.343888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54238 (local address 127.27.40.129:42471)
0809 03:36:23.344311 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:23.344315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:23.344322 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:36:23.344377 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:23.344379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:23.344397 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:23.344480 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:23.344756 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.345890 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.347253 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.347958 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.348018 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.348107 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:23.348957 (+ 850us) server_negotiation.cc:299] Negotiation successful
0809 03:36:23.349014 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":95,"threads_started":1}
I20260809 03:36:23.349284 20796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ff55e5a0af4dc19227aefe84c44779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:23.349248 20995 tablet_service.cc:1467] Processing CreateTablet for tablet f8ff55e5a0af4dc19227aefe84c44779 (DEFAULT_TABLE table=test-table [id=d3a88d36ffad46d2bbea0b0113b805ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:23.349619 20995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ff55e5a0af4dc19227aefe84c44779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:23.352898 21126 tablet_bootstrap.cc:492] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad: Bootstrap starting.
I20260809 03:36:23.353065 21125 tablet_bootstrap.cc:492] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba: Bootstrap starting.
I20260809 03:36:23.353771 21125 tablet_bootstrap.cc:654] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:23.353771 21126 tablet_bootstrap.cc:654] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:23.353489 20590 tablet_service.cc:1467] Processing CreateTablet for tablet f8ff55e5a0af4dc19227aefe84c44779 (DEFAULT_TABLE table=test-table [id=d3a88d36ffad46d2bbea0b0113b805ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:23.354118 20590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ff55e5a0af4dc19227aefe84c44779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:23.354425 21126 log.cc:824] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:23.354425 21125 log.cc:824] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:23.355243 21125 tablet_bootstrap.cc:492] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba: No bootstrap required, opened a new log
I20260809 03:36:23.355252 21126 tablet_bootstrap.cc:492] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad: No bootstrap required, opened a new log
I20260809 03:36:23.355322 21126 ts_tablet_manager.cc:1397] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:36:23.355322 21125 ts_tablet_manager.cc:1397] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:36:23.357308 21126 raft_consensus.cc:348] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db0768400157468b92f4c9032f9899ba" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45101 } } peers { permanent_uuid: "62c3debfae0e477e83c557fb65181bad" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 33111 } } peers { permanent_uuid: "e0b1d251e143403cae66634c4dc62b2d" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42471 } }
I20260809 03:36:23.357308 21125 raft_consensus.cc:348] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db0768400157468b92f4c9032f9899ba" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45101 } } peers { permanent_uuid: "62c3debfae0e477e83c557fb65181bad" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 33111 } } peers { permanent_uuid: "e0b1d251e143403cae66634c4dc62b2d" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42471 } }
I20260809 03:36:23.357501 21126 raft_consensus.cc:374] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:23.357501 21125 raft_consensus.cc:374] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:23.357573 21125 raft_consensus.cc:729] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db0768400157468b92f4c9032f9899ba, State: Initialized, Role: FOLLOWER
I20260809 03:36:23.357571 21126 raft_consensus.cc:729] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62c3debfae0e477e83c557fb65181bad, State: Initialized, Role: FOLLOWER
I20260809 03:36:23.357728 21126 consensus_queue.cc:260] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db0768400157468b92f4c9032f9899ba" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45101 } } peers { permanent_uuid: "62c3debfae0e477e83c557fb65181bad" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 33111 } } peers { permanent_uuid: "e0b1d251e143403cae66634c4dc62b2d" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42471 } }
I20260809 03:36:23.357756 21125 consensus_queue.cc:260] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db0768400157468b92f4c9032f9899ba" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45101 } } peers { permanent_uuid: "62c3debfae0e477e83c557fb65181bad" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 33111 } } peers { permanent_uuid: "e0b1d251e143403cae66634c4dc62b2d" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42471 } }
I20260809 03:36:23.357491 21131 tablet_bootstrap.cc:492] T f8ff55e5a0af4dc19227aefe84c44779 P e0b1d251e143403cae66634c4dc62b2d: Bootstrap starting.
I20260809 03:36:23.358040 21125 ts_tablet_manager.cc:1428] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:36:23.358039 21126 ts_tablet_manager.cc:1428] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:36:23.358167 21103 heartbeater.cc:503] Master 127.27.40.190:38813 was elected leader, sending a full tablet report...
I20260809 03:36:23.358279 20893 heartbeater.cc:503] Master 127.27.40.190:38813 was elected leader, sending a full tablet report...
I20260809 03:36:23.358672 21131 tablet_bootstrap.cc:654] T f8ff55e5a0af4dc19227aefe84c44779 P e0b1d251e143403cae66634c4dc62b2d: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:23.359366 21131 log.cc:824] T f8ff55e5a0af4dc19227aefe84c44779 P e0b1d251e143403cae66634c4dc62b2d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:23.360282 21131 tablet_bootstrap.cc:492] T f8ff55e5a0af4dc19227aefe84c44779 P e0b1d251e143403cae66634c4dc62b2d: No bootstrap required, opened a new log
I20260809 03:36:23.360349 21131 ts_tablet_manager.cc:1397] T f8ff55e5a0af4dc19227aefe84c44779 P e0b1d251e143403cae66634c4dc62b2d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:36:23.362036 21131 raft_consensus.cc:348] T f8ff55e5a0af4dc19227aefe84c44779 P e0b1d251e143403cae66634c4dc62b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db0768400157468b92f4c9032f9899ba" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45101 } } peers { permanent_uuid: "62c3debfae0e477e83c557fb65181bad" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 33111 } } peers { permanent_uuid: "e0b1d251e143403cae66634c4dc62b2d" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42471 } }
I20260809 03:36:23.362188 21131 raft_consensus.cc:374] T f8ff55e5a0af4dc19227aefe84c44779 P e0b1d251e143403cae66634c4dc62b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:23.362216 21131 raft_consensus.cc:729] T f8ff55e5a0af4dc19227aefe84c44779 P e0b1d251e143403cae66634c4dc62b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0b1d251e143403cae66634c4dc62b2d, State: Initialized, Role: FOLLOWER
I20260809 03:36:23.362381 21131 consensus_queue.cc:260] T f8ff55e5a0af4dc19227aefe84c44779 P e0b1d251e143403cae66634c4dc62b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db0768400157468b92f4c9032f9899ba" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45101 } } peers { permanent_uuid: "62c3debfae0e477e83c557fb65181bad" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 33111 } } peers { permanent_uuid: "e0b1d251e143403cae66634c4dc62b2d" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42471 } }
I20260809 03:36:23.362706 21131 ts_tablet_manager.cc:1428] T f8ff55e5a0af4dc19227aefe84c44779 P e0b1d251e143403cae66634c4dc62b2d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 03:36:23.362749 20689 heartbeater.cc:503] Master 127.27.40.190:38813 was elected leader, sending a full tablet report...
W20260809 03:36:23.367411 20894 tablet.cc:2369] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:36:23.424378 21133 raft_consensus.cc:482] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:36:23.424520 21133 raft_consensus.cc:504] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db0768400157468b92f4c9032f9899ba" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45101 } } peers { permanent_uuid: "62c3debfae0e477e83c557fb65181bad" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 33111 } } peers { permanent_uuid: "e0b1d251e143403cae66634c4dc62b2d" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42471 } }
I20260809 03:36:23.424916 21133 leader_election.cc:288] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62c3debfae0e477e83c557fb65181bad (127.27.40.131:33111), e0b1d251e143403cae66634c4dc62b2d (127.27.40.129:42471)
I20260809 03:36:23.428138 21124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.425187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:32797 (local address 127.27.40.131:33111)
0809 03:36:23.425244 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:23.425246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:23.425254 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:36:23.425282 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:23.425284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:23.425294 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:23.425358 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:23.425632 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.426184 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.427305 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.427727 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.427786 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.427794 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:23.428007 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 03:36:23.428052 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 03:36:23.428138 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.425080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:33111 (local address 127.27.40.130:32797)
0809 03:36:23.425142 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:23.425156 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:23.425205 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:23.425404 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:23.425405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:23.425408 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:23.425510 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.425512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.426281 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.426282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.427200 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.427203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.427821 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.427823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.427916 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.427920 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:36:23.427978 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 03:36:23.428043 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":4}
I20260809 03:36:23.428179 21121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.425222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:43457 (local address 127.27.40.129:42471)
0809 03:36:23.425287 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:23.425289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:23.425419 (+ 130us) server_negotiation.cc:407] Connection header received
0809 03:36:23.425454 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:23.425455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:23.425468 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:23.425573 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:23.425767 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.426313 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.427302 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.427791 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.427872 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.427881 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:23.428050 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 03:36:23.428104 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 03:36:23.428148 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.425120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:42471 (local address 127.27.40.130:43457)
0809 03:36:23.425333 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:23.425343 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:23.425397 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:23.425590 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:23.425592 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:23.425594 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:23.425684 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.425686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.426398 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.426399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.427200 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.427202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.427887 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.427887 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.427943 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.427947 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:36:23.428024 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 03:36:23.428063 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":7,"thread_start_us":152,"threads_started":1}
I20260809 03:36:23.428619 20617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8ff55e5a0af4dc19227aefe84c44779" candidate_uuid: "db0768400157468b92f4c9032f9899ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0b1d251e143403cae66634c4dc62b2d" is_pre_election: true
I20260809 03:36:23.428668 21026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8ff55e5a0af4dc19227aefe84c44779" candidate_uuid: "db0768400157468b92f4c9032f9899ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c3debfae0e477e83c557fb65181bad" is_pre_election: true
I20260809 03:36:23.428817 21026 raft_consensus.cc:2454] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db0768400157468b92f4c9032f9899ba in term 0.
I20260809 03:36:23.428817 20617 raft_consensus.cc:2454] T f8ff55e5a0af4dc19227aefe84c44779 P e0b1d251e143403cae66634c4dc62b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db0768400157468b92f4c9032f9899ba in term 0.
I20260809 03:36:23.429085 20737 leader_election.cc:302] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db0768400157468b92f4c9032f9899ba, e0b1d251e143403cae66634c4dc62b2d; no voters:
I20260809 03:36:23.429231 21133 raft_consensus.cc:2781] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:36:23.429291 21133 raft_consensus.cc:482] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:36:23.429329 21133 raft_consensus.cc:3037] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:23.430191 21133 raft_consensus.cc:504] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db0768400157468b92f4c9032f9899ba" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45101 } } peers { permanent_uuid: "62c3debfae0e477e83c557fb65181bad" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 33111 } } peers { permanent_uuid: "e0b1d251e143403cae66634c4dc62b2d" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42471 } }
I20260809 03:36:23.430346 21133 leader_election.cc:288] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [CANDIDATE]: Term 1 election: Requested vote from peers 62c3debfae0e477e83c557fb65181bad (127.27.40.131:33111), e0b1d251e143403cae66634c4dc62b2d (127.27.40.129:42471)
I20260809 03:36:23.430538 21026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8ff55e5a0af4dc19227aefe84c44779" candidate_uuid: "db0768400157468b92f4c9032f9899ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c3debfae0e477e83c557fb65181bad"
I20260809 03:36:23.430562 20617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8ff55e5a0af4dc19227aefe84c44779" candidate_uuid: "db0768400157468b92f4c9032f9899ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0b1d251e143403cae66634c4dc62b2d"
I20260809 03:36:23.430655 21026 raft_consensus.cc:3037] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:23.430676 20617 raft_consensus.cc:3037] T f8ff55e5a0af4dc19227aefe84c44779 P e0b1d251e143403cae66634c4dc62b2d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:23.431402 21026 raft_consensus.cc:2454] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db0768400157468b92f4c9032f9899ba in term 1.
I20260809 03:36:23.431541 20617 raft_consensus.cc:2454] T f8ff55e5a0af4dc19227aefe84c44779 P e0b1d251e143403cae66634c4dc62b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db0768400157468b92f4c9032f9899ba in term 1.
I20260809 03:36:23.431649 20729 leader_election.cc:302] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62c3debfae0e477e83c557fb65181bad, db0768400157468b92f4c9032f9899ba; no voters:
I20260809 03:36:23.431860 21133 raft_consensus.cc:2781] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:23.432053 21133 raft_consensus.cc:686] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [term 1 LEADER]: Becoming Leader. State: Replica: db0768400157468b92f4c9032f9899ba, State: Running, Role: LEADER
I20260809 03:36:23.432179 21133 consensus_queue.cc:237] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db0768400157468b92f4c9032f9899ba" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45101 } } peers { permanent_uuid: "62c3debfae0e477e83c557fb65181bad" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 33111 } } peers { permanent_uuid: "e0b1d251e143403cae66634c4dc62b2d" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42471 } }
I20260809 03:36:23.433099 20384 catalog_manager.cc:5495] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba reported cstate change: term changed from 0 to 1, leader changed from <none> to db0768400157468b92f4c9032f9899ba (127.27.40.130). New cstate: current_term: 1 leader_uuid: "db0768400157468b92f4c9032f9899ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db0768400157468b92f4c9032f9899ba" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62c3debfae0e477e83c557fb65181bad" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 33111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0b1d251e143403cae66634c4dc62b2d" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42471 } health_report { overall_health: UNKNOWN } } }
I20260809 03:36:23.444038 21120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.441585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41364 (local address 127.27.40.130:45101)
0809 03:36:23.441649 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:23.441652 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:23.441660 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:36:23.441695 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:23.441696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:23.441715 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:23.441839 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:36:23.441996 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.442466 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.443231 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.443443 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.443530 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.443700 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:36:23.443802 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:36:23.443903 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:23.443923 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 03:36:23.443942 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 03:36:23.445699 21026 raft_consensus.cc:1261] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad [term 1 FOLLOWER]: Refusing update from remote peer db0768400157468b92f4c9032f9899ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:36:23.445715 20617 raft_consensus.cc:1261] T f8ff55e5a0af4dc19227aefe84c44779 P e0b1d251e143403cae66634c4dc62b2d [term 1 FOLLOWER]: Refusing update from remote peer db0768400157468b92f4c9032f9899ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:36:23.446112 21140 consensus_queue.cc:1035] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "62c3debfae0e477e83c557fb65181bad" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 33111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:36:23.446300 21133 consensus_queue.cc:1035] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0b1d251e143403cae66634c4dc62b2d" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:36:23.449113 21147 mvcc.cc:205] Tried to move back new op lower bound from 7316466005791105024 to 7316466005738807296. Current Snapshot: MvccSnapshot[applied={T|T < 7316466005791105024}]
I20260809 03:36:23.449610 21146 mvcc.cc:205] Tried to move back new op lower bound from 7316466005791105024 to 7316466005738807296. Current Snapshot: MvccSnapshot[applied={T|T < 7316466005791105024}]
I20260809 03:36:23.451723 20384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:36:23.451810 20384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:36:23.452554 20384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 12b1c1b4e2a04750acb8ef4c98c9ddc7: Sending DeleteTablet for 3 replicas of tablet f8ff55e5a0af4dc19227aefe84c44779
I20260809 03:36:23.452860 20796 tablet_service.cc:1514] Processing DeleteTablet for tablet f8ff55e5a0af4dc19227aefe84c44779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:23 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:41362
I20260809 03:36:23.452883 20590 tablet_service.cc:1514] Processing DeleteTablet for tablet f8ff55e5a0af4dc19227aefe84c44779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:23 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:54238
I20260809 03:36:23.452942 20995 tablet_service.cc:1514] Processing DeleteTablet for tablet f8ff55e5a0af4dc19227aefe84c44779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:23 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:33704
I20260809 03:36:23.453099 21158 tablet_replica.cc:330] T f8ff55e5a0af4dc19227aefe84c44779 P e0b1d251e143403cae66634c4dc62b2d: stopping tablet replica
I20260809 03:36:23.453199 21158 raft_consensus.cc:2229] T f8ff55e5a0af4dc19227aefe84c44779 P e0b1d251e143403cae66634c4dc62b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:36:23.453269 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 20494
I20260809 03:36:23.462558 21159 tablet_replica.cc:330] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad: stopping tablet replica
I20260809 03:36:23.463266 21159 raft_consensus.cc:2229] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:36:23.463467 21159 raft_consensus.cc:2258] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:23.463704 21157 tablet_replica.cc:330] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba: stopping tablet replica
I20260809 03:36:23.463863 21157 raft_consensus.cc:2229] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:36:23.464057 21157 raft_consensus.cc:2258] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:23.467439 21159 ts_tablet_manager.cc:1905] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:36:23.467734 21157 ts_tablet_manager.cc:1905] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:36:23.469238 21159 ts_tablet_manager.cc:1918] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:36:23.469327 21159 log.cc:1196] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/f8ff55e5a0af4dc19227aefe84c44779
I20260809 03:36:23.469583 21157 ts_tablet_manager.cc:1918] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:36:23.469686 21159 ts_tablet_manager.cc:1939] T f8ff55e5a0af4dc19227aefe84c44779 P 62c3debfae0e477e83c557fb65181bad: Deleting consensus metadata
I20260809 03:36:23.469782 21157 log.cc:1196] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/f8ff55e5a0af4dc19227aefe84c44779
I20260809 03:36:23.470119 21157 ts_tablet_manager.cc:1939] T f8ff55e5a0af4dc19227aefe84c44779 P db0768400157468b92f4c9032f9899ba: Deleting consensus metadata
I20260809 03:36:23.470165 20365 catalog_manager.cc:4841] TS 62c3debfae0e477e83c557fb65181bad (127.27.40.131:33111): tablet f8ff55e5a0af4dc19227aefe84c44779 (table test-table [id=d3a88d36ffad46d2bbea0b0113b805ab]) successfully deleted
I20260809 03:36:23.470602 20365 catalog_manager.cc:4841] TS db0768400157468b92f4c9032f9899ba (127.27.40.130:45101): tablet f8ff55e5a0af4dc19227aefe84c44779 (table test-table [id=d3a88d36ffad46d2bbea0b0113b805ab]) successfully deleted
W20260809 03:36:23.472506 20369 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.40.129:42471 (error 108)
I20260809 03:36:23.472725 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 20695
I20260809 03:36:23.475683 21119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.475501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:42471 (local address 127.0.0.1:54248)
0809 03:36:23.475581 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:23.475639 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:42471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 03:36:23.475835 20369 catalog_manager.cc:4584] TS e0b1d251e143403cae66634c4dc62b2d (127.27.40.129:42471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f8ff55e5a0af4dc19227aefe84c44779: Network error: Client connection negotiation failed: client connection to 127.27.40.129:42471: connect: Connection refused (error 111)
I20260809 03:36:23.483042 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 20898
I20260809 03:36:23.495471 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 20340
2026-08-09T03:36:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2273 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T03:36:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:23Z Disabled control of system clock
I20260809 03:36:23.527901 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:38217
--webserver_interface=127.27.40.190
--webserver_port=0
--builtin_ntp_servers=127.27.40.148:45273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:38217
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:36:23.613078 21168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:23.613348 21168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:23.613373 21168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:23.614744 21168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:23.614801 21168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:23.614818 21168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:23.614835 21168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:23.614851 21168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:23.616292 21168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:45273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:38217
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:38217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:23.616516 21168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:23.616791 21168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:23.621873 21176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:23.622051 21168 server_base.cc:1034] running on GCE node
W20260809 03:36:23.621897 21177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:23.621873 21181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:23.622427 21168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:23.622704 21168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:23.623876 21168 hybrid_clock.cc:648] HybridClock initialized: now 1786246583623863 us; error 30 us; skew 500 ppm
I20260809 03:36:23.625206 21168 webserver.cc:460] Webserver started at http://127.27.40.190:38249/ using document root <none> and password file <none>
I20260809 03:36:23.625473 21168 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:23.625520 21168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:23.625757 21168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:23.626857 21168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "d8e5701b3e464376bfbbee1f23380581"
format_stamp: "Formatted at 2026-08-09 03:36:23 on dist-test-slave-gkw7"
server_key: "5add03584675b7d5fce0dda54e31f621"
server_key_iv: "58375a5aa681f5b8a16295450e03bfa4"
server_key_version: "encryptionkey@0"
I20260809 03:36:23.627259 21168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "d8e5701b3e464376bfbbee1f23380581"
format_stamp: "Formatted at 2026-08-09 03:36:23 on dist-test-slave-gkw7"
server_key: "5add03584675b7d5fce0dda54e31f621"
server_key_iv: "58375a5aa681f5b8a16295450e03bfa4"
server_key_version: "encryptionkey@0"
I20260809 03:36:23.628618 21168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:36:23.629426 21187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:23.629647 21168 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:36:23.629758 21168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "d8e5701b3e464376bfbbee1f23380581"
format_stamp: "Formatted at 2026-08-09 03:36:23 on dist-test-slave-gkw7"
server_key: "5add03584675b7d5fce0dda54e31f621"
server_key_iv: "58375a5aa681f5b8a16295450e03bfa4"
server_key_version: "encryptionkey@0"
I20260809 03:36:23.629834 21168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:23.643801 21168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:23.644140 21168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:23.644315 21168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:23.648597 21168 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:38217
I20260809 03:36:23.648638 21268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:38217 every 8 connection(s)
I20260809 03:36:23.649048 21168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:23.650261 21270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:23.652729 21270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581: Bootstrap starting.
I20260809 03:36:23.652830 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 21168
I20260809 03:36:23.652941 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:23.653155 27810 external_mini_cluster.cc:1428] Setting key 70f729726c5f9dffd6caf78f641bdc0b
I20260809 03:36:23.653465 21270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:23.653874 21270 log.cc:824] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:23.654706 21270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581: No bootstrap required, opened a new log
I20260809 03:36:23.656946 21270 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8e5701b3e464376bfbbee1f23380581" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38217 } }
I20260809 03:36:23.657141 21270 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:23.657181 21270 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8e5701b3e464376bfbbee1f23380581, State: Initialized, Role: FOLLOWER
I20260809 03:36:23.657351 21270 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8e5701b3e464376bfbbee1f23380581" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38217 } }
I20260809 03:36:23.657433 21270 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:23.657459 21270 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:23.657567 21270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:23.658383 21270 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8e5701b3e464376bfbbee1f23380581" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38217 } }
I20260809 03:36:23.658483 21270 leader_election.cc:302] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8e5701b3e464376bfbbee1f23380581; no voters:
I20260809 03:36:23.658720 21270 leader_election.cc:288] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:23.658754 21275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:23.658959 21275 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [term 1 LEADER]: Becoming Leader. State: Replica: d8e5701b3e464376bfbbee1f23380581, State: Running, Role: LEADER
I20260809 03:36:23.659058 21270 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:23.659067 21275 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8e5701b3e464376bfbbee1f23380581" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38217 } }
I20260809 03:36:23.659474 21275 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8e5701b3e464376bfbbee1f23380581. Latest consensus state: current_term: 1 leader_uuid: "d8e5701b3e464376bfbbee1f23380581" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8e5701b3e464376bfbbee1f23380581" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38217 } } }
I20260809 03:36:23.659566 21275 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:23.659709 21278 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8e5701b3e464376bfbbee1f23380581" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8e5701b3e464376bfbbee1f23380581" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38217 } } }
I20260809 03:36:23.659852 21278 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:23.659878 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.653711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43620 (local address 127.27.40.190:38217)
0809 03:36:23.653951 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:23.653975 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:23.654011 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:36:23.654586 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:23.654594 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:23.654677 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:23.654836 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:23.655137 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.656060 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.657274 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.657853 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.657948 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.658882 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:23.658886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:23.658886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:23.659111 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:23.659158 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:23.659174 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:23.659468 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:23.659504 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 03:36:23.659611 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":95,"threads_started":1}
I20260809 03:36:23.659993 21283 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:23.660512 21283 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:23.662066 21283 catalog_manager.cc:1347] Generated new cluster ID: e1bbad7b29664eb2b60df2a11477465e
I20260809 03:36:23.662118 21283 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:23.680877 21283 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:23.681465 21283 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:23.690515 21283 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581: Generated new TSK 0
I20260809 03:36:23.690753 21283 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:23.699131 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38217
--builtin_ntp_servers=127.27.40.148:45273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:36:23.783392 21302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:23.783595 21302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:23.783617 21302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:23.784848 21302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:23.784889 21302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:23.784914 21302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:36:23.786294 21302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:45273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:23.786537 21302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:23.786859 21302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:23.791975 21307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:23.792083 21302 server_base.cc:1034] running on GCE node
W20260809 03:36:23.791980 21308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:23.791971 21310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:23.792446 21302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:23.792696 21302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:23.793860 21302 hybrid_clock.cc:648] HybridClock initialized: now 1786246583793852 us; error 49 us; skew 500 ppm
I20260809 03:36:23.795217 21302 webserver.cc:460] Webserver started at http://127.27.40.129:46491/ using document root <none> and password file <none>
I20260809 03:36:23.795487 21302 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:23.795572 21302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:23.795742 21302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:23.796814 21302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "6903d85fc9eb4aeebd6a75c8f70c0f38"
format_stamp: "Formatted at 2026-08-09 03:36:23 on dist-test-slave-gkw7"
server_key: "0b61a7e0b91c68032d26e2d2957bc4ca"
server_key_iv: "3382cac63c48e814e6296977d2474aaa"
server_key_version: "encryptionkey@0"
I20260809 03:36:23.797191 21302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "6903d85fc9eb4aeebd6a75c8f70c0f38"
format_stamp: "Formatted at 2026-08-09 03:36:23 on dist-test-slave-gkw7"
server_key: "0b61a7e0b91c68032d26e2d2957bc4ca"
server_key_iv: "3382cac63c48e814e6296977d2474aaa"
server_key_version: "encryptionkey@0"
I20260809 03:36:23.798655 21302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:36:23.799419 21319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:23.799593 21302 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:36:23.799705 21302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "6903d85fc9eb4aeebd6a75c8f70c0f38"
format_stamp: "Formatted at 2026-08-09 03:36:23 on dist-test-slave-gkw7"
server_key: "0b61a7e0b91c68032d26e2d2957bc4ca"
server_key_iv: "3382cac63c48e814e6296977d2474aaa"
server_key_version: "encryptionkey@0"
I20260809 03:36:23.799783 21302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:23.816603 21302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:23.816946 21302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:23.817147 21302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:23.817392 21302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:23.817778 21302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:23.817836 21302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:23.817870 21302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:23.817901 21302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:23.824337 21302 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:38331
I20260809 03:36:23.824431 21486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:38331 every 8 connection(s)
I20260809 03:36:23.824769 21302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:36:23.825240 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 21302
I20260809 03:36:23.825357 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:36:23.825634 27810 external_mini_cluster.cc:1428] Setting key 214b8dca93364229070cc8f8bf51eee0
I20260809 03:36:23.826789 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38217
--builtin_ntp_servers=127.27.40.148:45273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 03:36:23.830763 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.825612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:49515 (local address 127.27.40.190:38217)
0809 03:36:23.825671 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:23.825674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:23.825950 (+ 276us) server_negotiation.cc:407] Connection header received
0809 03:36:23.826606 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:23.826608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:23.826622 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:23.826699 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:23.827453 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.828221 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.829224 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.829455 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.829537 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.830469 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:23.830471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:23.830472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:23.830477 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:23.830493 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:23.830497 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:23.830553 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:23.830653 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 03:36:23.830683 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 03:36:23.831071 21491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.825620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:38217 (local address 127.27.40.129:49515)
0809 03:36:23.825825 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:23.825868 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:23.826493 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:23.826757 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:23.826760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:23.826774 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:23.827363 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.827368 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.828324 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.828325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.829124 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.829126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.829567 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.829568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.829713 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.830323 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:36:23.830323 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:23.830353 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:36:23.830377 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:23.830558 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:23.830563 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:23.830566 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:36:23.830658 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 03:36:23.830780 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260809 03:36:23.831591 21489 heartbeater.cc:347] Connected to a master server at 127.27.40.190:38217
I20260809 03:36:23.831712 21489 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:23.831934 21489 heartbeater.cc:511] Master 127.27.40.190:38217 requested a full tablet report, sending...
I20260809 03:36:23.832554 21211 ts_manager.cc:194] Registered new tserver with Master: 6903d85fc9eb4aeebd6a75c8f70c0f38 (127.27.40.129:38331)
I20260809 03:36:23.833279 21211 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.40.129:49515
W20260809 03:36:23.910650 21492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:23.910845 21492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:23.910867 21492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:23.912146 21492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:23.912189 21492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:23.912214 21492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:36:23.913568 21492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:45273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:23.913813 21492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:23.914115 21492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:23.919064 21500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:23.919155 21506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:23.919113 21499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:23.919236 21492 server_base.cc:1034] running on GCE node
I20260809 03:36:23.919440 21492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:23.919740 21492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:23.920944 21492 hybrid_clock.cc:648] HybridClock initialized: now 1786246583920932 us; error 28 us; skew 500 ppm
I20260809 03:36:23.922362 21492 webserver.cc:460] Webserver started at http://127.27.40.130:42167/ using document root <none> and password file <none>
I20260809 03:36:23.922644 21492 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:23.922719 21492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:23.922871 21492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:23.923959 21492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "b0adb5a88f634b62a2498b648eee882b"
format_stamp: "Formatted at 2026-08-09 03:36:23 on dist-test-slave-gkw7"
server_key: "82489124312d38f1f4561f18e363a6bd"
server_key_iv: "9944c2b7e171c4ce38e3aec036108986"
server_key_version: "encryptionkey@0"
I20260809 03:36:23.924319 21492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "b0adb5a88f634b62a2498b648eee882b"
format_stamp: "Formatted at 2026-08-09 03:36:23 on dist-test-slave-gkw7"
server_key: "82489124312d38f1f4561f18e363a6bd"
server_key_iv: "9944c2b7e171c4ce38e3aec036108986"
server_key_version: "encryptionkey@0"
I20260809 03:36:23.925763 21492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:36:23.926568 21514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:23.926766 21492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:23.926874 21492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "b0adb5a88f634b62a2498b648eee882b"
format_stamp: "Formatted at 2026-08-09 03:36:23 on dist-test-slave-gkw7"
server_key: "82489124312d38f1f4561f18e363a6bd"
server_key_iv: "9944c2b7e171c4ce38e3aec036108986"
server_key_version: "encryptionkey@0"
I20260809 03:36:23.926949 21492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:23.946849 21492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:23.947208 21492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:23.947376 21492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:23.947626 21492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:23.947989 21492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:23.948047 21492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:23.948096 21492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:23.948122 21492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:23.954205 21492 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:44029
I20260809 03:36:23.954263 21682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:44029 every 8 connection(s)
I20260809 03:36:23.954594 21492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:36:23.960376 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.955448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:52519 (local address 127.27.40.190:38217)
0809 03:36:23.955497 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:23.955499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:23.955899 (+ 400us) server_negotiation.cc:407] Connection header received
0809 03:36:23.956504 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:23.956506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:23.956520 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:23.956574 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:23.957269 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.957878 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.958916 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.959125 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.959204 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.960065 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:23.960067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:23.960067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:23.960072 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:23.960088 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:23.960092 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:23.960167 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:23.960267 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 03:36:23.960296 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 03:36:23.960588 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:23.955454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:38217 (local address 127.27.40.130:52519)
0809 03:36:23.955719 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:23.955792 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:23.956390 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:23.956643 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:23.956647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:23.956657 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:23.957170 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.957175 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.957976 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.957977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.958820 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:23.958823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:23.959212 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:23.959213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:23.959357 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:23.959922 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:36:23.959922 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:23.959952 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:36:23.959975 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:23.960168 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:23.960175 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:23.960178 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:36:23.960269 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 03:36:23.960371 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":116,"threads_started":1}
I20260809 03:36:23.961143 21683 heartbeater.cc:347] Connected to a master server at 127.27.40.190:38217
I20260809 03:36:23.961243 21683 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:23.961429 21683 heartbeater.cc:511] Master 127.27.40.190:38217 requested a full tablet report, sending...
I20260809 03:36:23.961856 21211 ts_manager.cc:194] Registered new tserver with Master: b0adb5a88f634b62a2498b648eee882b (127.27.40.130:44029)
I20260809 03:36:23.962200 21211 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.40.130:52519
I20260809 03:36:23.963173 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 21492
I20260809 03:36:23.963297 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:36:23.963485 27810 external_mini_cluster.cc:1428] Setting key a862bb0e1b0712dbde7c3532c9498c97
I20260809 03:36:23.964350 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38217
--builtin_ntp_servers=127.27.40.148:45273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:36:24.046932 21687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:24.047137 21687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:24.047159 21687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:24.048316 21687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:24.048359 21687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:24.048385 21687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:36:24.049779 21687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:45273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:38217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:24.050035 21687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:24.050346 21687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:24.055429 21697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:24.055430 21702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:24.055470 21696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:24.055786 21687 server_base.cc:1034] running on GCE node
I20260809 03:36:24.056059 21687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:24.056322 21687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:24.057505 21687 hybrid_clock.cc:648] HybridClock initialized: now 1786246584057499 us; error 37 us; skew 500 ppm
I20260809 03:36:24.058913 21687 webserver.cc:460] Webserver started at http://127.27.40.131:41739/ using document root <none> and password file <none>
I20260809 03:36:24.059275 21687 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:24.059353 21687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:24.059474 21687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:24.060654 21687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "cbebdefa1d7a4d709cfbb3ad26581684"
format_stamp: "Formatted at 2026-08-09 03:36:24 on dist-test-slave-gkw7"
server_key: "48a3b8ff77a4a695435dbe75e8a34184"
server_key_iv: "177fb70d57d36be4122c81d0b85152b8"
server_key_version: "encryptionkey@0"
I20260809 03:36:24.060976 21687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "cbebdefa1d7a4d709cfbb3ad26581684"
format_stamp: "Formatted at 2026-08-09 03:36:24 on dist-test-slave-gkw7"
server_key: "48a3b8ff77a4a695435dbe75e8a34184"
server_key_iv: "177fb70d57d36be4122c81d0b85152b8"
server_key_version: "encryptionkey@0"
I20260809 03:36:24.062594 21687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 03:36:24.063326 21711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:24.063503 21687 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:36:24.063576 21687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "cbebdefa1d7a4d709cfbb3ad26581684"
format_stamp: "Formatted at 2026-08-09 03:36:24 on dist-test-slave-gkw7"
server_key: "48a3b8ff77a4a695435dbe75e8a34184"
server_key_iv: "177fb70d57d36be4122c81d0b85152b8"
server_key_version: "encryptionkey@0"
I20260809 03:36:24.063633 21687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:24.076162 21687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:24.076509 21687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:24.076629 21687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:24.076815 21687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:24.077170 21687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:24.077205 21687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:24.077229 21687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:24.077279 21687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:24.083536 21687 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:41665
I20260809 03:36:24.083609 21867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:41665 every 8 connection(s)
I20260809 03:36:24.083935 21687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:36:24.089679 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:24.084801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:54971 (local address 127.27.40.190:38217)
0809 03:36:24.084893 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:24.084895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:24.085123 (+ 228us) server_negotiation.cc:407] Connection header received
0809 03:36:24.085666 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:24.085667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:24.085679 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:24.085749 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:24.086421 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.086932 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.087960 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.088171 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.088244 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:24.089348 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:24.089349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:24.089350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:24.089354 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:24.089372 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:24.089375 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:24.089429 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:24.089518 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 03:36:24.089604 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:36:24.089875 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 21687
I20260809 03:36:24.089936 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:24.084753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:38217 (local address 127.27.40.131:54971)
0809 03:36:24.084988 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:24.085022 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:24.085510 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:24.085806 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:24.085810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:24.085818 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:24.086328 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:24.086332 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.087019 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.087020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:24.087859 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:24.087862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.088288 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.088290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:24.088409 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:24.089194 (+ 785us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:36:24.089195 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:24.089223 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:36:24.089246 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:24.089438 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:24.089442 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:24.089445 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:36:24.089497 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 03:36:24.089657 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":128,"threads_started":1}
I20260809 03:36:24.089993 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:36:24.090261 27810 external_mini_cluster.cc:1428] Setting key 628992d55d8e8cbf6977945fc2896bae
I20260809 03:36:24.090514 21869 heartbeater.cc:347] Connected to a master server at 127.27.40.190:38217
I20260809 03:36:24.090636 21869 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:24.090835 21869 heartbeater.cc:511] Master 127.27.40.190:38217 requested a full tablet report, sending...
I20260809 03:36:24.091496 21211 ts_manager.cc:194] Registered new tserver with Master: cbebdefa1d7a4d709cfbb3ad26581684 (127.27.40.131:41665)
I20260809 03:36:24.091902 21211 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.40.131:54971
I20260809 03:36:24.102492 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:36:24.107205 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:24.104159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43622 (local address 127.27.40.190:38217)
0809 03:36:24.104243 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:24.104245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:24.104342 (+ 97us) server_negotiation.cc:407] Connection header received
0809 03:36:24.104421 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:24.104422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:24.104433 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:24.104482 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:24.104647 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.105149 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.105997 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.106198 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.106277 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:24.106937 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:24.106938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:24.106939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:24.106944 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:24.106958 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:24.106962 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:24.107014 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:24.107096 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 03:36:24.107118 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 03:36:24.108994 21211 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:36:24.110754 21211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:36:24.116895 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:24.112958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:44029 (local address 127.0.0.1:51444)
0809 03:36:24.113074 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:24.113115 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:24.113175 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:24.113534 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:24.113538 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:24.113566 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:24.113638 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:24.113640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.114576 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.114578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:24.115629 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:24.115633 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.116583 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.116584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:24.116664 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:24.116678 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 03:36:24.116749 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 03:36:24.116816 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":5,"thread_start_us":71,"threads_started":1}
I20260809 03:36:24.117199 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:24.113086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51444 (local address 127.27.40.130:44029)
0809 03:36:24.113266 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:24.113272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:24.113284 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:36:24.113331 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:24.113333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:24.113351 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:24.113429 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:24.113736 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.114491 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.115722 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.116379 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.116449 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:24.116545 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:24.117066 (+ 521us) server_negotiation.cc:299] Negotiation successful
0809 03:36:24.117109 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
I20260809 03:36:24.117075 21880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:24.113046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:38331 (local address 127.0.0.1:34708)
0809 03:36:24.113294 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:24.113305 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:24.113355 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:24.113843 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:24.113844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:24.113846 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:24.113897 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:24.113898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.114797 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.114797 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:24.115924 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:24.115927 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.116843 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.116844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:24.116903 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:24.116907 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:36:24.116965 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 03:36:24.117001 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":81,"threads_started":1}
I20260809 03:36:24.117348 21885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:24.113424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34708 (local address 127.27.40.129:38331)
0809 03:36:24.113690 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:24.113695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:24.113705 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:36:24.113744 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:24.113746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:24.113761 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:24.113845 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:24.113955 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.114724 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.116013 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.116760 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.116826 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:24.116914 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:24.117132 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 03:36:24.117175 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":80,"threads_started":1}
I20260809 03:36:24.118095 21585 tablet_service.cc:1467] Processing CreateTablet for tablet c6fd34158bea47a6aecaffc8d7fcb7a3 (DEFAULT_TABLE table=test-table [id=102c5373f3514c8790dbcd729ae020dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:24.118180 21388 tablet_service.cc:1467] Processing CreateTablet for tablet c6fd34158bea47a6aecaffc8d7fcb7a3 (DEFAULT_TABLE table=test-table [id=102c5373f3514c8790dbcd729ae020dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:24.118499 21585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6fd34158bea47a6aecaffc8d7fcb7a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:24.118547 21388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6fd34158bea47a6aecaffc8d7fcb7a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:24.118932 21882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:24.113181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:41665 (local address 127.0.0.1:53948)
0809 03:36:24.114597 (+ 1416us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:24.114606 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:24.114680 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:24.114869 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:24.114870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:24.114875 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:24.114910 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:24.114912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.116322 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.116323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:24.117146 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:24.117151 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.118686 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.118687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:24.118746 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:24.118750 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:36:24.118804 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 03:36:24.118842 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1402,"thread_start_us":72,"threads_started":1}
I20260809 03:36:24.119230 21883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:24.113390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53948 (local address 127.27.40.131:41665)
0809 03:36:24.113618 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:24.113623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:24.114677 (+ 1054us) server_negotiation.cc:407] Connection header received
0809 03:36:24.114778 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:24.114781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:24.114797 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:24.114871 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:24.114969 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.116238 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.117239 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.118491 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.118563 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:24.118647 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:24.119088 (+ 441us) server_negotiation.cc:299] Negotiation successful
0809 03:36:24.119129 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":103,"threads_started":1}
I20260809 03:36:24.119866 21776 tablet_service.cc:1467] Processing CreateTablet for tablet c6fd34158bea47a6aecaffc8d7fcb7a3 (DEFAULT_TABLE table=test-table [id=102c5373f3514c8790dbcd729ae020dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:24.120204 21776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6fd34158bea47a6aecaffc8d7fcb7a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:24.121443 21886 tablet_bootstrap.cc:492] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b: Bootstrap starting.
I20260809 03:36:24.122087 21886 tablet_bootstrap.cc:654] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:24.122289 21889 tablet_bootstrap.cc:492] T c6fd34158bea47a6aecaffc8d7fcb7a3 P 6903d85fc9eb4aeebd6a75c8f70c0f38: Bootstrap starting.
I20260809 03:36:24.122439 21886 log.cc:824] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:24.122954 21889 tablet_bootstrap.cc:654] T c6fd34158bea47a6aecaffc8d7fcb7a3 P 6903d85fc9eb4aeebd6a75c8f70c0f38: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:24.123265 21889 log.cc:824] T c6fd34158bea47a6aecaffc8d7fcb7a3 P 6903d85fc9eb4aeebd6a75c8f70c0f38: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:24.123270 21886 tablet_bootstrap.cc:492] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b: No bootstrap required, opened a new log
I20260809 03:36:24.123354 21886 ts_tablet_manager.cc:1397] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:36:24.123557 21892 tablet_bootstrap.cc:492] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684: Bootstrap starting.
I20260809 03:36:24.124163 21892 tablet_bootstrap.cc:654] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:24.124456 21892 log.cc:824] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:24.125160 21892 tablet_bootstrap.cc:492] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684: No bootstrap required, opened a new log
I20260809 03:36:24.125180 21889 tablet_bootstrap.cc:492] T c6fd34158bea47a6aecaffc8d7fcb7a3 P 6903d85fc9eb4aeebd6a75c8f70c0f38: No bootstrap required, opened a new log
I20260809 03:36:24.125232 21892 ts_tablet_manager.cc:1397] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:36:24.125232 21889 ts_tablet_manager.cc:1397] T c6fd34158bea47a6aecaffc8d7fcb7a3 P 6903d85fc9eb4aeebd6a75c8f70c0f38: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 03:36:24.125711 21886 raft_consensus.cc:348] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0adb5a88f634b62a2498b648eee882b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44029 } } peers { permanent_uuid: "6903d85fc9eb4aeebd6a75c8f70c0f38" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38331 } } peers { permanent_uuid: "cbebdefa1d7a4d709cfbb3ad26581684" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 41665 } }
I20260809 03:36:24.125857 21886 raft_consensus.cc:374] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:24.125900 21886 raft_consensus.cc:729] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0adb5a88f634b62a2498b648eee882b, State: Initialized, Role: FOLLOWER
I20260809 03:36:24.126015 21886 consensus_queue.cc:260] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0adb5a88f634b62a2498b648eee882b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44029 } } peers { permanent_uuid: "6903d85fc9eb4aeebd6a75c8f70c0f38" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38331 } } peers { permanent_uuid: "cbebdefa1d7a4d709cfbb3ad26581684" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 41665 } }
I20260809 03:36:24.126264 21886 ts_tablet_manager.cc:1428] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:36:24.126410 21683 heartbeater.cc:503] Master 127.27.40.190:38217 was elected leader, sending a full tablet report...
I20260809 03:36:24.127786 21889 raft_consensus.cc:348] T c6fd34158bea47a6aecaffc8d7fcb7a3 P 6903d85fc9eb4aeebd6a75c8f70c0f38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0adb5a88f634b62a2498b648eee882b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44029 } } peers { permanent_uuid: "6903d85fc9eb4aeebd6a75c8f70c0f38" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38331 } } peers { permanent_uuid: "cbebdefa1d7a4d709cfbb3ad26581684" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 41665 } }
I20260809 03:36:24.127786 21892 raft_consensus.cc:348] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0adb5a88f634b62a2498b648eee882b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44029 } } peers { permanent_uuid: "6903d85fc9eb4aeebd6a75c8f70c0f38" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38331 } } peers { permanent_uuid: "cbebdefa1d7a4d709cfbb3ad26581684" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 41665 } }
I20260809 03:36:24.127944 21892 raft_consensus.cc:374] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:24.127944 21889 raft_consensus.cc:374] T c6fd34158bea47a6aecaffc8d7fcb7a3 P 6903d85fc9eb4aeebd6a75c8f70c0f38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:24.127981 21892 raft_consensus.cc:729] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbebdefa1d7a4d709cfbb3ad26581684, State: Initialized, Role: FOLLOWER
I20260809 03:36:24.127982 21889 raft_consensus.cc:729] T c6fd34158bea47a6aecaffc8d7fcb7a3 P 6903d85fc9eb4aeebd6a75c8f70c0f38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6903d85fc9eb4aeebd6a75c8f70c0f38, State: Initialized, Role: FOLLOWER
I20260809 03:36:24.128098 21892 consensus_queue.cc:260] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0adb5a88f634b62a2498b648eee882b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44029 } } peers { permanent_uuid: "6903d85fc9eb4aeebd6a75c8f70c0f38" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38331 } } peers { permanent_uuid: "cbebdefa1d7a4d709cfbb3ad26581684" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 41665 } }
I20260809 03:36:24.128098 21889 consensus_queue.cc:260] T c6fd34158bea47a6aecaffc8d7fcb7a3 P 6903d85fc9eb4aeebd6a75c8f70c0f38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0adb5a88f634b62a2498b648eee882b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44029 } } peers { permanent_uuid: "6903d85fc9eb4aeebd6a75c8f70c0f38" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38331 } } peers { permanent_uuid: "cbebdefa1d7a4d709cfbb3ad26581684" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 41665 } }
I20260809 03:36:24.128353 21889 ts_tablet_manager.cc:1428] T c6fd34158bea47a6aecaffc8d7fcb7a3 P 6903d85fc9eb4aeebd6a75c8f70c0f38: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:36:24.128353 21892 ts_tablet_manager.cc:1428] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 03:36:24.128405 21489 heartbeater.cc:503] Master 127.27.40.190:38217 was elected leader, sending a full tablet report...
I20260809 03:36:24.128520 21869 heartbeater.cc:503] Master 127.27.40.190:38217 was elected leader, sending a full tablet report...
W20260809 03:36:24.205122 21684 tablet.cc:2369] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:36:24.206758 21896 raft_consensus.cc:482] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:36:24.206895 21896 raft_consensus.cc:504] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0adb5a88f634b62a2498b648eee882b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44029 } } peers { permanent_uuid: "6903d85fc9eb4aeebd6a75c8f70c0f38" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38331 } } peers { permanent_uuid: "cbebdefa1d7a4d709cfbb3ad26581684" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 41665 } }
I20260809 03:36:24.207336 21896 leader_election.cc:288] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0adb5a88f634b62a2498b648eee882b (127.27.40.130:44029), 6903d85fc9eb4aeebd6a75c8f70c0f38 (127.27.40.129:38331)
I20260809 03:36:24.210445 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:24.207490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:38331 (local address 127.27.40.131:55869)
0809 03:36:24.207546 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:24.207570 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:24.207615 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:24.207871 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:24.207872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:24.207874 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:24.207958 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:24.207960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.208841 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.208843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:24.209617 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:24.209619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.210186 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.210187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:24.210261 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:24.210269 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 03:36:24.210330 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 03:36:24.210375 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 03:36:24.210444 21885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:24.207689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:55869 (local address 127.27.40.129:38331)
0809 03:36:24.207753 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:24.207755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:24.207763 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:36:24.207789 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:24.207790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:24.207802 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:24.207862 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:24.208046 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.208743 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.209710 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.210094 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.210178 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:24.210187 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:24.210336 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 03:36:24.210375 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 03:36:24.210911 21418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6fd34158bea47a6aecaffc8d7fcb7a3" candidate_uuid: "cbebdefa1d7a4d709cfbb3ad26581684" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6903d85fc9eb4aeebd6a75c8f70c0f38" is_pre_election: true
I20260809 03:36:24.211037 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:24.207601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:47455 (local address 127.27.40.130:44029)
0809 03:36:24.207645 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:24.207647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:24.208348 (+ 701us) server_negotiation.cc:407] Connection header received
0809 03:36:24.208420 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:24.208421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:24.208433 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:24.208479 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:24.208620 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.209070 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.210052 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.210687 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.210771 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:24.210781 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:24.210930 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 03:36:24.210971 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 03:36:24.211038 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:24.207502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:44029 (local address 127.27.40.131:47455)
0809 03:36:24.208261 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:24.208272 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:24.208353 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:24.208479 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:24.208480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:24.208481 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:24.208550 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:24.208551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.209152 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.209153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:24.209964 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:24.209966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.210778 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.210779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:24.210848 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:24.210852 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:36:24.210909 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 03:36:24.210943 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":107,"threads_started":1}
I20260809 03:36:24.211079 21418 raft_consensus.cc:2454] T c6fd34158bea47a6aecaffc8d7fcb7a3 P 6903d85fc9eb4aeebd6a75c8f70c0f38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbebdefa1d7a4d709cfbb3ad26581684 in term 0.
I20260809 03:36:24.211356 21712 leader_election.cc:302] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6903d85fc9eb4aeebd6a75c8f70c0f38, cbebdefa1d7a4d709cfbb3ad26581684; no voters:
I20260809 03:36:24.211467 21615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6fd34158bea47a6aecaffc8d7fcb7a3" candidate_uuid: "cbebdefa1d7a4d709cfbb3ad26581684" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0adb5a88f634b62a2498b648eee882b" is_pre_election: true
I20260809 03:36:24.211500 21896 raft_consensus.cc:2781] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:36:24.211566 21896 raft_consensus.cc:482] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:36:24.211588 21896 raft_consensus.cc:3037] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:24.211629 21615 raft_consensus.cc:2454] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbebdefa1d7a4d709cfbb3ad26581684 in term 0.
I20260809 03:36:24.212505 21896 raft_consensus.cc:504] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0adb5a88f634b62a2498b648eee882b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44029 } } peers { permanent_uuid: "6903d85fc9eb4aeebd6a75c8f70c0f38" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38331 } } peers { permanent_uuid: "cbebdefa1d7a4d709cfbb3ad26581684" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 41665 } }
I20260809 03:36:24.212642 21896 leader_election.cc:288] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [CANDIDATE]: Term 1 election: Requested vote from peers b0adb5a88f634b62a2498b648eee882b (127.27.40.130:44029), 6903d85fc9eb4aeebd6a75c8f70c0f38 (127.27.40.129:38331)
I20260809 03:36:24.212774 21615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6fd34158bea47a6aecaffc8d7fcb7a3" candidate_uuid: "cbebdefa1d7a4d709cfbb3ad26581684" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0adb5a88f634b62a2498b648eee882b"
I20260809 03:36:24.212800 21418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6fd34158bea47a6aecaffc8d7fcb7a3" candidate_uuid: "cbebdefa1d7a4d709cfbb3ad26581684" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6903d85fc9eb4aeebd6a75c8f70c0f38"
I20260809 03:36:24.212849 21615 raft_consensus.cc:3037] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:24.212888 21418 raft_consensus.cc:3037] T c6fd34158bea47a6aecaffc8d7fcb7a3 P 6903d85fc9eb4aeebd6a75c8f70c0f38 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:24.213639 21418 raft_consensus.cc:2454] T c6fd34158bea47a6aecaffc8d7fcb7a3 P 6903d85fc9eb4aeebd6a75c8f70c0f38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbebdefa1d7a4d709cfbb3ad26581684 in term 1.
I20260809 03:36:24.213920 21615 raft_consensus.cc:2454] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbebdefa1d7a4d709cfbb3ad26581684 in term 1.
I20260809 03:36:24.214094 21713 leader_election.cc:302] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0adb5a88f634b62a2498b648eee882b, cbebdefa1d7a4d709cfbb3ad26581684; no voters:
I20260809 03:36:24.214190 21896 raft_consensus.cc:2781] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:24.214372 21896 raft_consensus.cc:686] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [term 1 LEADER]: Becoming Leader. State: Replica: cbebdefa1d7a4d709cfbb3ad26581684, State: Running, Role: LEADER
I20260809 03:36:24.214474 21896 consensus_queue.cc:237] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0adb5a88f634b62a2498b648eee882b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44029 } } peers { permanent_uuid: "6903d85fc9eb4aeebd6a75c8f70c0f38" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38331 } } peers { permanent_uuid: "cbebdefa1d7a4d709cfbb3ad26581684" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 41665 } }
I20260809 03:36:24.215299 21210 catalog_manager.cc:5495] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 reported cstate change: term changed from 0 to 1, leader changed from <none> to cbebdefa1d7a4d709cfbb3ad26581684 (127.27.40.131). New cstate: current_term: 1 leader_uuid: "cbebdefa1d7a4d709cfbb3ad26581684" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0adb5a88f634b62a2498b648eee882b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6903d85fc9eb4aeebd6a75c8f70c0f38" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbebdefa1d7a4d709cfbb3ad26581684" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 41665 } health_report { overall_health: HEALTHY } } }
I20260809 03:36:24.235422 21883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:24.232686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53956 (local address 127.27.40.131:41665)
0809 03:36:24.232783 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:24.232785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:24.232795 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:36:24.232843 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:24.232844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:24.232867 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:24.232954 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:36:24.233173 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.233737 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.234559 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:24.234762 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:24.234857 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:24.235040 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:36:24.235150 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:36:24.235251 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:24.235276 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 03:36:24.235298 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:36:24.237458 21615 raft_consensus.cc:1261] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b [term 1 FOLLOWER]: Refusing update from remote peer cbebdefa1d7a4d709cfbb3ad26581684: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:36:24.237458 21418 raft_consensus.cc:1261] T c6fd34158bea47a6aecaffc8d7fcb7a3 P 6903d85fc9eb4aeebd6a75c8f70c0f38 [term 1 FOLLOWER]: Refusing update from remote peer cbebdefa1d7a4d709cfbb3ad26581684: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:36:24.238021 21896 consensus_queue.cc:1035] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6903d85fc9eb4aeebd6a75c8f70c0f38" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:36:24.238205 21901 consensus_queue.cc:1035] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0adb5a88f634b62a2498b648eee882b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 44029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:36:24.240713 21906 mvcc.cc:205] Tried to move back new op lower bound from 7316466009033781248 to 7316466008943243264. Current Snapshot: MvccSnapshot[applied={T|T < 7316466009033781248}]
I20260809 03:36:24.241050 21905 mvcc.cc:205] Tried to move back new op lower bound from 7316466009033781248 to 7316466008943243264. Current Snapshot: MvccSnapshot[applied={T|T < 7316466009033781248}]
I20260809 03:36:24.241580 21907 mvcc.cc:205] Tried to move back new op lower bound from 7316466009033781248 to 7316466008943243264. Current Snapshot: MvccSnapshot[applied={T|T < 7316466009033781248}]
I20260809 03:36:24.243430 21211 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:36:24.243532 21211 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:36:24.244319 21211 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d8e5701b3e464376bfbbee1f23380581: Sending DeleteTablet for 3 replicas of tablet c6fd34158bea47a6aecaffc8d7fcb7a3
I20260809 03:36:24.244793 21388 tablet_service.cc:1514] Processing DeleteTablet for tablet c6fd34158bea47a6aecaffc8d7fcb7a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:24 UTC) from {username='slave'} at 127.0.0.1:34708
I20260809 03:36:24.244827 21585 tablet_service.cc:1514] Processing DeleteTablet for tablet c6fd34158bea47a6aecaffc8d7fcb7a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:24 UTC) from {username='slave'} at 127.0.0.1:51444
I20260809 03:36:24.244966 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 21302
I20260809 03:36:24.245067 21776 tablet_service.cc:1514] Processing DeleteTablet for tablet c6fd34158bea47a6aecaffc8d7fcb7a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:24 UTC) from {username='slave'} at 127.0.0.1:53948
I20260809 03:36:24.251784 21918 tablet_replica.cc:330] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684: stopping tablet replica
I20260809 03:36:24.251977 21918 raft_consensus.cc:2229] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:36:24.252151 21918 raft_consensus.cc:2258] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:24.252936 21916 tablet_replica.cc:330] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b: stopping tablet replica
I20260809 03:36:24.253057 21916 raft_consensus.cc:2229] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:36:24.253160 21918 ts_tablet_manager.cc:1905] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:36:24.253183 21916 raft_consensus.cc:2258] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:24.253644 21916 ts_tablet_manager.cc:1905] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:36:24.255174 21918 ts_tablet_manager.cc:1918] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:36:24.255266 21918 log.cc:1196] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/c6fd34158bea47a6aecaffc8d7fcb7a3
I20260809 03:36:24.255462 21916 ts_tablet_manager.cc:1918] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:36:24.255555 21916 log.cc:1196] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/c6fd34158bea47a6aecaffc8d7fcb7a3
I20260809 03:36:24.255594 21918 ts_tablet_manager.cc:1939] T c6fd34158bea47a6aecaffc8d7fcb7a3 P cbebdefa1d7a4d709cfbb3ad26581684: Deleting consensus metadata
I20260809 03:36:24.255831 21916 ts_tablet_manager.cc:1939] T c6fd34158bea47a6aecaffc8d7fcb7a3 P b0adb5a88f634b62a2498b648eee882b: Deleting consensus metadata
I20260809 03:36:24.256053 21189 catalog_manager.cc:4841] TS cbebdefa1d7a4d709cfbb3ad26581684 (127.27.40.131:41665): tablet c6fd34158bea47a6aecaffc8d7fcb7a3 (table test-table [id=102c5373f3514c8790dbcd729ae020dc]) successfully deleted
I20260809 03:36:24.256393 21189 catalog_manager.cc:4841] TS b0adb5a88f634b62a2498b648eee882b (127.27.40.130:44029): tablet c6fd34158bea47a6aecaffc8d7fcb7a3 (table test-table [id=102c5373f3514c8790dbcd729ae020dc]) successfully deleted
W20260809 03:36:24.257514 21188 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.40.129:38331 (error 108)
I20260809 03:36:24.257866 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 21492
I20260809 03:36:24.258123 21882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:24.257999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:38331 (local address 127.0.0.1:34720)
0809 03:36:24.258047 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:24.258084 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:38331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 03:36:24.258220 21188 catalog_manager.cc:4584] TS 6903d85fc9eb4aeebd6a75c8f70c0f38 (127.27.40.129:38331): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c6fd34158bea47a6aecaffc8d7fcb7a3: Network error: Client connection negotiation failed: client connection to 127.27.40.129:38331: connect: Connection refused (error 111)
I20260809 03:36:24.267921 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 21687
I20260809 03:36:24.281210 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 21168
2026-08-09T03:36:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (784 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:24 dist-test-slave-gkw7 krb5kdc[21926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:24 dist-test-slave-gkw7 krb5kdc[21926](info): set up 2 sockets
Aug 09 03:36:24 dist-test-slave-gkw7 krb5kdc[21926](info): commencing operation
krb5kdc: starting...
W20260809 03:36:25.511979 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:25 dist-test-slave-gkw7 krb5kdc[21926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:36:25.640784 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:45989
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:45989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:25.723364 21942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:25.723574 21942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:25.723614 21942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:25.724680 21942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:25.724714 21942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:25.724728 21942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:25.724741 21942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:25.724753 21942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:25.726167 21942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:45989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:45989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:25.726361 21942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:25.726521 21942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:25.731361 21950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:25.731521 21942 server_base.cc:1034] running on GCE node
W20260809 03:36:25.731364 21947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:25.731364 21948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:25.732142 21942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:25.732443 21942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:25.733597 21942 hybrid_clock.cc:648] HybridClock initialized: now 1786246585733586 us; error 44 us; skew 500 ppm
Aug 09 03:36:25 dist-test-slave-gkw7 krb5kdc[21926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246585, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:25.736168 21942 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:36:25.736579 21942 webserver.cc:460] Webserver started at http://127.27.40.190:44083/ using document root <none> and password file <none>
I20260809 03:36:25.736836 21942 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:25.736905 21942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:25.737058 21942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:25.738159 21942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "b16c16d23cdf4512be60cdce2c2f290c"
format_stamp: "Formatted at 2026-08-09 03:36:25 on dist-test-slave-gkw7"
server_key: "45a7e28e2e6f9d61e19dd69cb5ada5da"
server_key_iv: "008ad9376a6e9b4327aa605e91e067c1"
server_key_version: "encryptionkey@0"
I20260809 03:36:25.738520 21942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "b16c16d23cdf4512be60cdce2c2f290c"
format_stamp: "Formatted at 2026-08-09 03:36:25 on dist-test-slave-gkw7"
server_key: "45a7e28e2e6f9d61e19dd69cb5ada5da"
server_key_iv: "008ad9376a6e9b4327aa605e91e067c1"
server_key_version: "encryptionkey@0"
I20260809 03:36:25.739869 21942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:36:25.740628 21959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:25.740814 21942 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:25.740921 21942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "b16c16d23cdf4512be60cdce2c2f290c"
format_stamp: "Formatted at 2026-08-09 03:36:25 on dist-test-slave-gkw7"
server_key: "45a7e28e2e6f9d61e19dd69cb5ada5da"
server_key_iv: "008ad9376a6e9b4327aa605e91e067c1"
server_key_version: "encryptionkey@0"
I20260809 03:36:25.740993 21942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:25.770129 21942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:25.772104 21942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:25.772279 21942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:25.776427 21942 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:45989
I20260809 03:36:25.776464 22043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:45989 every 8 connection(s)
I20260809 03:36:25.776840 21942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:25.777803 22044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:25.780058 22044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c: Bootstrap starting.
I20260809 03:36:25.780598 22044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:25.780860 22044 log.cc:824] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:25.781519 22044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c: No bootstrap required, opened a new log
I20260809 03:36:25.783151 22044 raft_consensus.cc:348] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16c16d23cdf4512be60cdce2c2f290c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 45989 } }
I20260809 03:36:25.783308 22044 raft_consensus.cc:374] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:25.783329 22044 raft_consensus.cc:729] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b16c16d23cdf4512be60cdce2c2f290c, State: Initialized, Role: FOLLOWER
I20260809 03:36:25.783463 22044 consensus_queue.cc:260] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16c16d23cdf4512be60cdce2c2f290c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 45989 } }
I20260809 03:36:25.783552 22044 raft_consensus.cc:388] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:25.783576 22044 raft_consensus.cc:482] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:25.783603 22044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:25.784080 22044 raft_consensus.cc:504] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16c16d23cdf4512be60cdce2c2f290c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 45989 } }
I20260809 03:36:25.784171 22044 leader_election.cc:302] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b16c16d23cdf4512be60cdce2c2f290c; no voters:
I20260809 03:36:25.784294 22044 leader_election.cc:288] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:25.784365 22048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:25.784550 22048 raft_consensus.cc:686] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [term 1 LEADER]: Becoming Leader. State: Replica: b16c16d23cdf4512be60cdce2c2f290c, State: Running, Role: LEADER
I20260809 03:36:25.784574 22044 sys_catalog.cc:564] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:25.784669 22048 consensus_queue.cc:237] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16c16d23cdf4512be60cdce2c2f290c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 45989 } }
I20260809 03:36:25.785257 22050 sys_catalog.cc:455] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b16c16d23cdf4512be60cdce2c2f290c. Latest consensus state: current_term: 1 leader_uuid: "b16c16d23cdf4512be60cdce2c2f290c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16c16d23cdf4512be60cdce2c2f290c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 45989 } } }
I20260809 03:36:25.785362 22050 sys_catalog.cc:458] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:25.785563 22049 sys_catalog.cc:455] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b16c16d23cdf4512be60cdce2c2f290c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16c16d23cdf4512be60cdce2c2f290c" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 45989 } } }
I20260809 03:36:25.785686 22049 sys_catalog.cc:458] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:25.785712 22059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:25.786258 22059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:25.786384 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 21942
I20260809 03:36:25.786489 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:25.786695 27810 external_mini_cluster.cc:1428] Setting key 6f8dc8a40445b74bcbb7fcb69f878ff0
I20260809 03:36:25.787890 22059 catalog_manager.cc:1347] Generated new cluster ID: 4e35170c1cff41fda1e4a0b785c86865
I20260809 03:36:25.787943 22059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:25.791302 22059 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:25.791990 22059 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 03:36:25 dist-test-slave-gkw7 krb5kdc[21926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:25.798770 22074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:25.787220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55066 (local address 127.27.40.190:45989)
0809 03:36:25.787378 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:25.787386 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:25.787514 (+ 128us) server_negotiation.cc:407] Connection header received
0809 03:36:25.787655 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:25.787670 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:25.787725 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:25.787875 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:25.788812 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:25.790281 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:25.791381 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:25.792001 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:25.792116 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:25.795022 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:25.795026 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:25.795027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:25.795039 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:25.797049 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:25.797478 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:25.797481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:25.797481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:25.797626 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:25.797847 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:25.797848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:25.797849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:25.798133 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:25.798286 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:25.798380 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 03:36:25.798530 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260809 03:36:25.800787 22059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c: Generated new TSK 0
I20260809 03:36:25.800963 22059 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:36:25.832058 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:45989
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:25.913978 22078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:25.914176 22078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:25.914199 22078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:25.915403 22078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:25.915450 22078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:25.915477 22078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:36:25.917042 22078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:45989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:25.917297 22078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:25.917570 22078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:25.922595 22087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:25.922631 22084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:25.922628 22083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:25.923343 22078 server_base.cc:1034] running on GCE node
I20260809 03:36:25.923514 22078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:25.923700 22078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:25.924834 22078 hybrid_clock.cc:648] HybridClock initialized: now 1786246585924826 us; error 37 us; skew 500 ppm
Aug 09 03:36:25 dist-test-slave-gkw7 krb5kdc[21926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246585, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:25.927531 22078 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:36:25.927994 22078 webserver.cc:460] Webserver started at http://127.27.40.129:35999/ using document root <none> and password file <none>
I20260809 03:36:25.928186 22078 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:25.928226 22078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:25.928315 22078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:25.929284 22078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "ffca517d6ce84a00b4438485d429e894"
format_stamp: "Formatted at 2026-08-09 03:36:25 on dist-test-slave-gkw7"
server_key: "b738bb5aeef35f61860b8976b732332b"
server_key_iv: "92c6d1eaa3cc59eb27ce60b9e5398be6"
server_key_version: "encryptionkey@0"
I20260809 03:36:25.929656 22078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "ffca517d6ce84a00b4438485d429e894"
format_stamp: "Formatted at 2026-08-09 03:36:25 on dist-test-slave-gkw7"
server_key: "b738bb5aeef35f61860b8976b732332b"
server_key_iv: "92c6d1eaa3cc59eb27ce60b9e5398be6"
server_key_version: "encryptionkey@0"
I20260809 03:36:25.930964 22078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:25.931651 22095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:25.931838 22078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:25.931942 22078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "ffca517d6ce84a00b4438485d429e894"
format_stamp: "Formatted at 2026-08-09 03:36:25 on dist-test-slave-gkw7"
server_key: "b738bb5aeef35f61860b8976b732332b"
server_key_iv: "92c6d1eaa3cc59eb27ce60b9e5398be6"
server_key_version: "encryptionkey@0"
I20260809 03:36:25.932015 22078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:25.951467 22078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:25.953868 22078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:25.954033 22078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:25.954284 22078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:25.954628 22078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:25.954685 22078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:25.954725 22078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:25.954753 22078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:25.960641 22078 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:38701
I20260809 03:36:25.960695 22251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:38701 every 8 connection(s)
I20260809 03:36:25.961025 22078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
Aug 09 03:36:25 dist-test-slave-gkw7 krb5kdc[21926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246585, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:25.967748 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 22078
I20260809 03:36:25.967896 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:36:25.968175 27810 external_mini_cluster.cc:1428] Setting key 9d129170c4d9754bac21a35c9d181901
I20260809 03:36:25.971316 22074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:25.961816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:60633 (local address 127.27.40.190:45989)
0809 03:36:25.961867 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:25.961869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:25.962088 (+ 219us) server_negotiation.cc:407] Connection header received
0809 03:36:25.962617 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:25.962619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:25.962633 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:25.962687 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:25.963650 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:25.964177 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:25.965185 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:25.965400 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:25.965488 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:25.967689 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:25.967690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:25.967691 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:25.967699 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:25.969489 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:25.970074 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:25.970075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:25.970076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:25.970144 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:25.970466 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:25.970466 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:25.970467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:25.970586 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:25.970644 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:25.971034 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 03:36:25.971169 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 03:36:25.971386 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:25.961793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:45989 (local address 127.27.40.129:60633)
0809 03:36:25.961966 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:25.962002 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:25.962511 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:25.962768 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:25.962771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:25.963340 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:25.963528 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:25.963532 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:25.964285 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:25.964286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:25.965092 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:25.965094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:25.965508 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:25.965509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:25.965658 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:25.966144 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:25.966152 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:25.967584 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:25.969633 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:25.969641 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:25.969644 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:25.969986 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:25.970240 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:25.970240 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:25.970241 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:25.970394 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:25.970653 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:25.970659 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:25.970838 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 03:36:25.970959 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 03:36:25.971167 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260809 03:36:25.972100 22252 heartbeater.cc:347] Connected to a master server at 127.27.40.190:45989
I20260809 03:36:25.972229 22252 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:25.972476 22252 heartbeater.cc:511] Master 127.27.40.190:45989 requested a full tablet report, sending...
I20260809 03:36:25.973177 21987 ts_manager.cc:194] Registered new tserver with Master: ffca517d6ce84a00b4438485d429e894 (127.27.40.129:38701)
I20260809 03:36:25.974185 21987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:60633
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:36:26.000079 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:45989
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:26.085189 22260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:26.085423 22260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:26.085464 22260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:26.086701 22260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:26.086771 22260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:26.086814 22260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:36:26.088285 22260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:45989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:26.088563 22260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:26.088801 22260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:26.094041 22267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:26.094038 22268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:26.094143 22270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:26.094213 22260 server_base.cc:1034] running on GCE node
I20260809 03:36:26.094457 22260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:26.094663 22260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:26.095799 22260 hybrid_clock.cc:648] HybridClock initialized: now 1786246586095774 us; error 37 us; skew 500 ppm
Aug 09 03:36:26 dist-test-slave-gkw7 krb5kdc[21926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246586, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:26.098419 22260 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:36:26.098817 22260 webserver.cc:460] Webserver started at http://127.27.40.130:42509/ using document root <none> and password file <none>
I20260809 03:36:26.099022 22260 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:26.099063 22260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:26.099156 22260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:26.100170 22260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "65257b0b1ce34839be1bddc8098cf6af"
format_stamp: "Formatted at 2026-08-09 03:36:26 on dist-test-slave-gkw7"
server_key: "ab09242081be0f49271b85fe716eddb5"
server_key_iv: "ce5eebfb4f2a6955b26ed0bd85258f39"
server_key_version: "encryptionkey@0"
I20260809 03:36:26.100481 22260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "65257b0b1ce34839be1bddc8098cf6af"
format_stamp: "Formatted at 2026-08-09 03:36:26 on dist-test-slave-gkw7"
server_key: "ab09242081be0f49271b85fe716eddb5"
server_key_iv: "ce5eebfb4f2a6955b26ed0bd85258f39"
server_key_version: "encryptionkey@0"
I20260809 03:36:26.101853 22260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 03:36:26.102679 22277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:26.102890 22260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:26.102977 22260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "65257b0b1ce34839be1bddc8098cf6af"
format_stamp: "Formatted at 2026-08-09 03:36:26 on dist-test-slave-gkw7"
server_key: "ab09242081be0f49271b85fe716eddb5"
server_key_iv: "ce5eebfb4f2a6955b26ed0bd85258f39"
server_key_version: "encryptionkey@0"
I20260809 03:36:26.103035 22260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:26.117100 22260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:26.119724 22260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:26.119855 22260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:26.120056 22260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:26.120329 22260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:26.120363 22260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:26.120388 22260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:26.120404 22260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:26.126754 22260 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:42439
I20260809 03:36:26.126839 22426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:42439 every 8 connection(s)
I20260809 03:36:26.127203 22260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
Aug 09 03:36:26 dist-test-slave-gkw7 krb5kdc[21926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246586, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:26.136685 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 22260
I20260809 03:36:26.136812 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:36:26.136938 22074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.128046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:49635 (local address 127.27.40.190:45989)
0809 03:36:26.128130 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:26.128132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:26.128372 (+ 240us) server_negotiation.cc:407] Connection header received
0809 03:36:26.128888 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:26.128889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:26.128902 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:26.128973 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:26.130001 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.130513 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.131510 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.131712 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.131770 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.133963 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:26.133965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:26.133966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:26.133973 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:26.135500 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:26.135929 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:26.135930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:26.135930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:26.135985 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:26.136209 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:26.136210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:26.136210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:26.136317 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:26.136369 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:26.136707 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 03:36:26.136813 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:36:26.137035 27810 external_mini_cluster.cc:1428] Setting key 81230e0aab9425630d31afd45b44f79f
I20260809 03:36:26.137151 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.128039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:45989 (local address 127.27.40.130:49635)
0809 03:36:26.128238 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:26.128281 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:26.128771 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:26.129028 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:26.129031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:26.129707 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:26.129911 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:26.129916 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.130623 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.130624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:26.131423 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:26.131425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.131790 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.131791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:26.131895 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.132368 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:26.132379 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:26.133861 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:26.135577 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:26.135584 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:26.135586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:26.135863 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:26.136048 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:26.136049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:26.136049 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:26.136150 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:26.136375 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:26.136380 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:26.136540 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 03:36:26.136689 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 03:36:26.136880 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":126,"threads_started":1}
I20260809 03:36:26.137710 22427 heartbeater.cc:347] Connected to a master server at 127.27.40.190:45989
I20260809 03:36:26.137837 22427 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:26.138042 22427 heartbeater.cc:511] Master 127.27.40.190:45989 requested a full tablet report, sending...
I20260809 03:36:26.138420 21987 ts_manager.cc:194] Registered new tserver with Master: 65257b0b1ce34839be1bddc8098cf6af (127.27.40.130:42439)
I20260809 03:36:26.138819 21987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:49635
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:36:26.163981 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:45989
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:26.248368 22434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:26.248575 22434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:26.248601 22434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:26.249737 22434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:26.249773 22434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:26.249799 22434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:36:26.251195 22434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:45989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:26.251386 22434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:26.251564 22434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:26.256707 22440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:26.256781 22442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:26.256934 22434 server_base.cc:1034] running on GCE node
W20260809 03:36:26.256973 22439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:26.257237 22434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:26.257480 22434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:26.258663 22434 hybrid_clock.cc:648] HybridClock initialized: now 1786246586258652 us; error 32 us; skew 500 ppm
Aug 09 03:36:26 dist-test-slave-gkw7 krb5kdc[21926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246586, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:26.261158 22434 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:36:26.261714 22434 webserver.cc:460] Webserver started at http://127.27.40.131:35333/ using document root <none> and password file <none>
I20260809 03:36:26.262013 22434 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:26.262090 22434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:26.262233 22434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:26.263278 22434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "b02a1d2b233b44daae6ed472429a3921"
format_stamp: "Formatted at 2026-08-09 03:36:26 on dist-test-slave-gkw7"
server_key: "42d773fc43dd77c059e509d67529d304"
server_key_iv: "2f8a6fa53e14a0b1fcf19d5ef11badba"
server_key_version: "encryptionkey@0"
I20260809 03:36:26.263654 22434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "b02a1d2b233b44daae6ed472429a3921"
format_stamp: "Formatted at 2026-08-09 03:36:26 on dist-test-slave-gkw7"
server_key: "42d773fc43dd77c059e509d67529d304"
server_key_iv: "2f8a6fa53e14a0b1fcf19d5ef11badba"
server_key_version: "encryptionkey@0"
I20260809 03:36:26.264988 22434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:36:26.265738 22453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:26.265923 22434 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 03:36:26.266016 22434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "b02a1d2b233b44daae6ed472429a3921"
format_stamp: "Formatted at 2026-08-09 03:36:26 on dist-test-slave-gkw7"
server_key: "42d773fc43dd77c059e509d67529d304"
server_key_iv: "2f8a6fa53e14a0b1fcf19d5ef11badba"
server_key_version: "encryptionkey@0"
I20260809 03:36:26.266072 22434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:26.272994 22434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:26.276024 22434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:26.276225 22434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:26.276451 22434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:26.276875 22434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:26.276942 22434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:26.276976 22434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:26.277004 22434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:26.288420 22434 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:34913
I20260809 03:36:26.288465 22617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:34913 every 8 connection(s)
I20260809 03:36:26.288813 22434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:36:26.289449 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 22434
I20260809 03:36:26.289599 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:36:26.289840 27810 external_mini_cluster.cc:1428] Setting key 68fd59d669f75dea73cf23fc5f03f92e
Aug 09 03:36:26 dist-test-slave-gkw7 krb5kdc[21926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246586, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:26.307600 22074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.289698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:36189 (local address 127.27.40.190:45989)
0809 03:36:26.289763 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:26.289766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:26.290288 (+ 522us) server_negotiation.cc:407] Connection header received
0809 03:36:26.290815 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:26.290816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:26.290829 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:26.290879 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:26.292330 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.292840 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.294617 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.295034 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.295085 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.303661 (+ 8576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:26.303663 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:26.303664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:26.303673 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:26.305668 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:26.306257 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:26.306258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:26.306259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:26.306336 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:26.306701 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:26.306702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:26.306702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:26.306861 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:26.306924 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:26.307379 (+ 455us) server_negotiation.cc:299] Negotiation successful
0809 03:36:26.307477 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 03:36:26.307977 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.289828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:45989 (local address 127.27.40.131:36189)
0809 03:36:26.290160 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:26.290206 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:26.290705 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:26.291018 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:26.291022 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:26.291881 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:26.292219 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:26.292227 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.292971 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.292972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:26.294496 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:26.294500 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.295225 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.295227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:26.295412 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.301639 (+ 6227us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:26.301651 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:26.303551 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:26.305779 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:26.305795 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:26.305798 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:26.306147 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:26.306429 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:26.306430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:26.306431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:26.306611 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:26.306940 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:26.306947 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:26.307176 (+ 229us) client_negotiation.cc:769] Sending connection context
0809 03:36:26.307361 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 03:36:26.307606 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":269,"threads_started":1}
I20260809 03:36:26.308523 22619 heartbeater.cc:347] Connected to a master server at 127.27.40.190:45989
I20260809 03:36:26.308671 22619 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:26.308940 22619 heartbeater.cc:511] Master 127.27.40.190:45989 requested a full tablet report, sending...
I20260809 03:36:26.309391 21987 ts_manager.cc:194] Registered new tserver with Master: b02a1d2b233b44daae6ed472429a3921 (127.27.40.131:34913)
I20260809 03:36:26.309988 21987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:36189
I20260809 03:36:26.312439 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:36:26.319809 22074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.313766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56242 (local address 127.27.40.190:45989)
0809 03:36:26.313803 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:26.313805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:26.313897 (+ 92us) server_negotiation.cc:407] Connection header received
0809 03:36:26.313951 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:26.313951 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:26.313959 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:26.314009 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:26.314513 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.314974 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.315687 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.315860 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.315908 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.317294 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:26.317295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:26.317296 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:26.317302 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:26.318737 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:26.319100 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:26.319101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:26.319101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:26.319151 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:26.319400 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:26.319401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:26.319401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:26.319518 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:26.319596 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:26.319704 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 03:36:26.319752 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:36:26.321595 21987 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:36:26.323167 21987 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:36:26.329810 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.325473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:34913 (local address 127.0.0.1:45388)
0809 03:36:26.325697 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:26.325744 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:26.325816 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:26.326176 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:26.326180 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:26.326182 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:26.326273 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:26.326275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.327700 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.327701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:26.328614 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:26.328617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.329450 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.329451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:26.329567 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.329582 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 03:36:26.329659 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 03:36:26.329710 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":99,"threads_started":1}
I20260809 03:36:26.329909 22637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.325483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:42439 (local address 127.0.0.1:60154)
0809 03:36:26.326192 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:26.326202 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:26.326252 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:26.326520 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:26.326520 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:26.326522 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:26.326574 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:26.326576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.327408 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.327409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:26.328689 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:26.328691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.329632 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.329633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:26.329695 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.329698 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:36:26.329775 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 03:36:26.329806 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":134,"threads_started":1}
I20260809 03:36:26.330058 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.325482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:38701 (local address 127.0.0.1:49392)
0809 03:36:26.325965 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:26.325976 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:26.326016 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:26.326390 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:26.326391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:26.326393 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:26.326430 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:26.326432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.327717 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.327718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:26.328823 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:26.328826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.329810 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.329811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:26.329868 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.329871 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:36:26.329929 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 03:36:26.329982 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":45,"thread_start_us":156,"threads_started":1}
I20260809 03:36:26.330190 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.325716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45388 (local address 127.27.40.131:34913)
0809 03:36:26.325914 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:26.325919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:26.325932 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:36:26.325973 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:26.325975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:26.325995 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:26.326091 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:26.326373 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.327564 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.328765 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.329354 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.329437 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.329494 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:26.330052 (+ 558us) server_negotiation.cc:299] Negotiation successful
0809 03:36:26.330095 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":95,"threads_started":1}
I20260809 03:36:26.330251 22641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.325805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60154 (local address 127.27.40.130:42439)
0809 03:36:26.326379 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:26.326383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:26.326393 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:36:26.326431 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:26.326433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:26.326448 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:26.326524 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:26.326631 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.327309 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.328789 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.329525 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.329631 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.329698 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:26.329791 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 03:36:26.329835 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":73,"threads_started":1}
I20260809 03:36:26.330535 22643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.325820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49392 (local address 127.27.40.129:38701)
0809 03:36:26.325975 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:26.325979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:26.326096 (+ 117us) server_negotiation.cc:407] Connection header received
0809 03:36:26.326144 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:26.326146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:26.326162 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:26.326253 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:26.326516 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.327618 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.328910 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.329634 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.329704 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.329773 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:26.330159 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 03:36:26.330206 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260809 03:36:26.330896 22517 tablet_service.cc:1467] Processing CreateTablet for tablet 421b16fc42504de7b8b048158cf500cf (DEFAULT_TABLE table=test-table [id=80eac886cb3b461e8bc03b0e48dbb0a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:26.330917 22344 tablet_service.cc:1467] Processing CreateTablet for tablet 421b16fc42504de7b8b048158cf500cf (DEFAULT_TABLE table=test-table [id=80eac886cb3b461e8bc03b0e48dbb0a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:26.331260 22517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421b16fc42504de7b8b048158cf500cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:26.331161 22164 tablet_service.cc:1467] Processing CreateTablet for tablet 421b16fc42504de7b8b048158cf500cf (DEFAULT_TABLE table=test-table [id=80eac886cb3b461e8bc03b0e48dbb0a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:26.331262 22344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421b16fc42504de7b8b048158cf500cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:26.331458 22164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421b16fc42504de7b8b048158cf500cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:26.334671 22647 tablet_bootstrap.cc:492] T 421b16fc42504de7b8b048158cf500cf P ffca517d6ce84a00b4438485d429e894: Bootstrap starting.
I20260809 03:36:26.334758 22645 tablet_bootstrap.cc:492] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921: Bootstrap starting.
I20260809 03:36:26.335374 22645 tablet_bootstrap.cc:654] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:26.335404 22647 tablet_bootstrap.cc:654] T 421b16fc42504de7b8b048158cf500cf P ffca517d6ce84a00b4438485d429e894: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:26.335901 22645 log.cc:824] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:26.335940 22647 log.cc:824] T 421b16fc42504de7b8b048158cf500cf P ffca517d6ce84a00b4438485d429e894: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:26.336306 22652 tablet_bootstrap.cc:492] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af: Bootstrap starting.
I20260809 03:36:26.336664 22645 tablet_bootstrap.cc:492] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921: No bootstrap required, opened a new log
I20260809 03:36:26.336745 22645 ts_tablet_manager.cc:1397] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:36:26.336964 22652 tablet_bootstrap.cc:654] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:26.337438 22652 log.cc:824] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:26.337918 22647 tablet_bootstrap.cc:492] T 421b16fc42504de7b8b048158cf500cf P ffca517d6ce84a00b4438485d429e894: No bootstrap required, opened a new log
I20260809 03:36:26.337993 22647 ts_tablet_manager.cc:1397] T 421b16fc42504de7b8b048158cf500cf P ffca517d6ce84a00b4438485d429e894: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:36:26.338207 22652 tablet_bootstrap.cc:492] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af: No bootstrap required, opened a new log
I20260809 03:36:26.338287 22652 ts_tablet_manager.cc:1397] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:36:26.338737 22645 raft_consensus.cc:348] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffca517d6ce84a00b4438485d429e894" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38701 } } peers { permanent_uuid: "b02a1d2b233b44daae6ed472429a3921" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34913 } } peers { permanent_uuid: "65257b0b1ce34839be1bddc8098cf6af" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 42439 } }
I20260809 03:36:26.338919 22645 raft_consensus.cc:374] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:26.338959 22645 raft_consensus.cc:729] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b02a1d2b233b44daae6ed472429a3921, State: Initialized, Role: FOLLOWER
I20260809 03:36:26.339135 22645 consensus_queue.cc:260] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffca517d6ce84a00b4438485d429e894" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38701 } } peers { permanent_uuid: "b02a1d2b233b44daae6ed472429a3921" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34913 } } peers { permanent_uuid: "65257b0b1ce34839be1bddc8098cf6af" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 42439 } }
I20260809 03:36:26.339457 22645 ts_tablet_manager.cc:1428] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:36:26.339599 22619 heartbeater.cc:503] Master 127.27.40.190:45989 was elected leader, sending a full tablet report...
I20260809 03:36:26.340067 22647 raft_consensus.cc:348] T 421b16fc42504de7b8b048158cf500cf P ffca517d6ce84a00b4438485d429e894 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffca517d6ce84a00b4438485d429e894" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38701 } } peers { permanent_uuid: "b02a1d2b233b44daae6ed472429a3921" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34913 } } peers { permanent_uuid: "65257b0b1ce34839be1bddc8098cf6af" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 42439 } }
I20260809 03:36:26.340253 22647 raft_consensus.cc:374] T 421b16fc42504de7b8b048158cf500cf P ffca517d6ce84a00b4438485d429e894 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:26.340173 22652 raft_consensus.cc:348] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffca517d6ce84a00b4438485d429e894" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38701 } } peers { permanent_uuid: "b02a1d2b233b44daae6ed472429a3921" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34913 } } peers { permanent_uuid: "65257b0b1ce34839be1bddc8098cf6af" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 42439 } }
I20260809 03:36:26.340322 22647 raft_consensus.cc:729] T 421b16fc42504de7b8b048158cf500cf P ffca517d6ce84a00b4438485d429e894 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffca517d6ce84a00b4438485d429e894, State: Initialized, Role: FOLLOWER
I20260809 03:36:26.340361 22652 raft_consensus.cc:374] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:26.340418 22652 raft_consensus.cc:729] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65257b0b1ce34839be1bddc8098cf6af, State: Initialized, Role: FOLLOWER
I20260809 03:36:26.340543 22652 consensus_queue.cc:260] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffca517d6ce84a00b4438485d429e894" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38701 } } peers { permanent_uuid: "b02a1d2b233b44daae6ed472429a3921" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34913 } } peers { permanent_uuid: "65257b0b1ce34839be1bddc8098cf6af" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 42439 } }
I20260809 03:36:26.340579 22647 consensus_queue.cc:260] T 421b16fc42504de7b8b048158cf500cf P ffca517d6ce84a00b4438485d429e894 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffca517d6ce84a00b4438485d429e894" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38701 } } peers { permanent_uuid: "b02a1d2b233b44daae6ed472429a3921" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34913 } } peers { permanent_uuid: "65257b0b1ce34839be1bddc8098cf6af" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 42439 } }
I20260809 03:36:26.340842 22427 heartbeater.cc:503] Master 127.27.40.190:45989 was elected leader, sending a full tablet report...
I20260809 03:36:26.340845 22652 ts_tablet_manager.cc:1428] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:36:26.341007 22647 ts_tablet_manager.cc:1428] T 421b16fc42504de7b8b048158cf500cf P ffca517d6ce84a00b4438485d429e894: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:36:26.341066 22252 heartbeater.cc:503] Master 127.27.40.190:45989 was elected leader, sending a full tablet report...
W20260809 03:36:26.377892 22428 tablet.cc:2369] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:36:26.461771 22253 tablet.cc:2369] T 421b16fc42504de7b8b048158cf500cf P ffca517d6ce84a00b4438485d429e894: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:36:26.525947 22658 raft_consensus.cc:482] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:36:26.526086 22658 raft_consensus.cc:504] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffca517d6ce84a00b4438485d429e894" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38701 } } peers { permanent_uuid: "b02a1d2b233b44daae6ed472429a3921" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34913 } } peers { permanent_uuid: "65257b0b1ce34839be1bddc8098cf6af" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 42439 } }
I20260809 03:36:26.526487 22658 leader_election.cc:288] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffca517d6ce84a00b4438485d429e894 (127.27.40.129:38701), 65257b0b1ce34839be1bddc8098cf6af (127.27.40.130:42439)
I20260809 03:36:26.529497 22643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.526765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:50211 (local address 127.27.40.129:38701)
0809 03:36:26.526816 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:26.526819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:26.526835 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:36:26.526926 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:26.526927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:26.526944 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:26.527018 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:26.527184 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.527672 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.528751 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.529129 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.529220 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.529230 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:26.529391 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 03:36:26.529432 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 03:36:26.529497 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.526717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:38701 (local address 127.27.40.131:50211)
0809 03:36:26.526785 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:26.526797 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:26.526845 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:26.527022 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:26.527023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:26.527025 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:26.527108 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:26.527110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.527767 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.527768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:26.528642 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:26.528644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.529218 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.529219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:26.529302 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.529305 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:36:26.529381 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 03:36:26.529421 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 03:36:26.529614 22641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.526809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:37309 (local address 127.27.40.130:42439)
0809 03:36:26.526897 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:26.526899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:26.527030 (+ 131us) server_negotiation.cc:407] Connection header received
0809 03:36:26.527117 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:26.527118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:26.527128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:26.527219 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:26.527390 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.527851 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.528807 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.529196 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.529282 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.529291 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:26.529448 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 03:36:26.529490 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 03:36:26.529633 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.526727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:42439 (local address 127.27.40.131:37309)
0809 03:36:26.526959 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:26.526967 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:26.527047 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:26.527231 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:26.527232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:26.527233 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:26.527310 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:26.527311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.527936 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.527938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:26.528707 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:26.528732 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.529282 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.529284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:26.529360 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.529373 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 03:36:26.529443 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 03:36:26.529483 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":24,"thread_start_us":125,"threads_started":1}
I20260809 03:36:26.529951 22188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421b16fc42504de7b8b048158cf500cf" candidate_uuid: "b02a1d2b233b44daae6ed472429a3921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffca517d6ce84a00b4438485d429e894" is_pre_election: true
I20260809 03:36:26.530037 22368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421b16fc42504de7b8b048158cf500cf" candidate_uuid: "b02a1d2b233b44daae6ed472429a3921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65257b0b1ce34839be1bddc8098cf6af" is_pre_election: true
I20260809 03:36:26.530122 22188 raft_consensus.cc:2454] T 421b16fc42504de7b8b048158cf500cf P ffca517d6ce84a00b4438485d429e894 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b02a1d2b233b44daae6ed472429a3921 in term 0.
I20260809 03:36:26.530211 22368 raft_consensus.cc:2454] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b02a1d2b233b44daae6ed472429a3921 in term 0.
I20260809 03:36:26.530387 22458 leader_election.cc:302] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b02a1d2b233b44daae6ed472429a3921, ffca517d6ce84a00b4438485d429e894; no voters:
I20260809 03:36:26.530579 22658 raft_consensus.cc:2781] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:36:26.530620 22658 raft_consensus.cc:482] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:36:26.530637 22658 raft_consensus.cc:3037] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:26.531313 22658 raft_consensus.cc:504] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffca517d6ce84a00b4438485d429e894" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38701 } } peers { permanent_uuid: "b02a1d2b233b44daae6ed472429a3921" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34913 } } peers { permanent_uuid: "65257b0b1ce34839be1bddc8098cf6af" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 42439 } }
I20260809 03:36:26.531457 22658 leader_election.cc:288] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [CANDIDATE]: Term 1 election: Requested vote from peers ffca517d6ce84a00b4438485d429e894 (127.27.40.129:38701), 65257b0b1ce34839be1bddc8098cf6af (127.27.40.130:42439)
I20260809 03:36:26.531603 22368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421b16fc42504de7b8b048158cf500cf" candidate_uuid: "b02a1d2b233b44daae6ed472429a3921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65257b0b1ce34839be1bddc8098cf6af"
I20260809 03:36:26.531620 22188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421b16fc42504de7b8b048158cf500cf" candidate_uuid: "b02a1d2b233b44daae6ed472429a3921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffca517d6ce84a00b4438485d429e894"
I20260809 03:36:26.531682 22368 raft_consensus.cc:3037] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:26.531701 22188 raft_consensus.cc:3037] T 421b16fc42504de7b8b048158cf500cf P ffca517d6ce84a00b4438485d429e894 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:26.532397 22188 raft_consensus.cc:2454] T 421b16fc42504de7b8b048158cf500cf P ffca517d6ce84a00b4438485d429e894 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b02a1d2b233b44daae6ed472429a3921 in term 1.
I20260809 03:36:26.532505 22368 raft_consensus.cc:2454] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b02a1d2b233b44daae6ed472429a3921 in term 1.
I20260809 03:36:26.532625 22458 leader_election.cc:302] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b02a1d2b233b44daae6ed472429a3921, ffca517d6ce84a00b4438485d429e894; no voters:
I20260809 03:36:26.532765 22658 raft_consensus.cc:2781] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:26.532938 22658 raft_consensus.cc:686] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [term 1 LEADER]: Becoming Leader. State: Replica: b02a1d2b233b44daae6ed472429a3921, State: Running, Role: LEADER
I20260809 03:36:26.533056 22658 consensus_queue.cc:237] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffca517d6ce84a00b4438485d429e894" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38701 } } peers { permanent_uuid: "b02a1d2b233b44daae6ed472429a3921" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34913 } } peers { permanent_uuid: "65257b0b1ce34839be1bddc8098cf6af" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 42439 } }
I20260809 03:36:26.533933 21987 catalog_manager.cc:5495] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 reported cstate change: term changed from 0 to 1, leader changed from <none> to b02a1d2b233b44daae6ed472429a3921 (127.27.40.131). New cstate: current_term: 1 leader_uuid: "b02a1d2b233b44daae6ed472429a3921" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffca517d6ce84a00b4438485d429e894" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b02a1d2b233b44daae6ed472429a3921" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65257b0b1ce34839be1bddc8098cf6af" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 42439 } health_report { overall_health: UNKNOWN } } }
W20260809 03:36:26.540187 22620 tablet.cc:2369] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:36:26.559362 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.557092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45404 (local address 127.27.40.131:34913)
0809 03:36:26.557173 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:26.557175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:26.557191 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:36:26.557223 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:26.557223 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:26.557240 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:26.557300 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:36:26.557435 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.557909 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.558656 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.558834 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.558883 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.559005 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:36:26.559088 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:36:26.559164 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:26.559222 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 03:36:26.559231 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:36:26.561223 22368 raft_consensus.cc:1261] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af [term 1 FOLLOWER]: Refusing update from remote peer b02a1d2b233b44daae6ed472429a3921: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:36:26.561223 22188 raft_consensus.cc:1261] T 421b16fc42504de7b8b048158cf500cf P ffca517d6ce84a00b4438485d429e894 [term 1 FOLLOWER]: Refusing update from remote peer b02a1d2b233b44daae6ed472429a3921: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:36:26.561631 22664 consensus_queue.cc:1035] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffca517d6ce84a00b4438485d429e894" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:36:26.561838 22658 consensus_queue.cc:1035] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65257b0b1ce34839be1bddc8098cf6af" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 42439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:36:26.566030 21985 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:36:26.566125 21985 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:36:26.566958 21985 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b16c16d23cdf4512be60cdce2c2f290c: Sending DeleteTablet for 3 replicas of tablet 421b16fc42504de7b8b048158cf500cf
I20260809 03:36:26.567304 22343 tablet_service.cc:1514] Processing DeleteTablet for tablet 421b16fc42504de7b8b048158cf500cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:26 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:60154
I20260809 03:36:26.567371 22344 tablet_service.cc:1514] Processing DeleteTablet for tablet 421b16fc42504de7b8b048158cf500cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:26 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:60154
I20260809 03:36:26.567476 22517 tablet_service.cc:1514] Processing DeleteTablet for tablet 421b16fc42504de7b8b048158cf500cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:26 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:45388
I20260809 03:36:26.567545 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 22078
I20260809 03:36:26.578053 22682 tablet_replica.cc:330] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921: stopping tablet replica
I20260809 03:36:26.578171 22682 raft_consensus.cc:2229] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:36:26.578267 22682 raft_consensus.cc:2258] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:26.579545 22682 ts_tablet_manager.cc:1905] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:36:26.581338 22681 tablet_replica.cc:330] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af: stopping tablet replica
I20260809 03:36:26.581465 22681 raft_consensus.cc:2229] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:36:26.581601 22681 raft_consensus.cc:2258] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:26.581838 22682 ts_tablet_manager.cc:1918] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:36:26.581907 22682 log.cc:1196] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/421b16fc42504de7b8b048158cf500cf
I20260809 03:36:26.582199 22682 ts_tablet_manager.cc:1939] T 421b16fc42504de7b8b048158cf500cf P b02a1d2b233b44daae6ed472429a3921: Deleting consensus metadata
I20260809 03:36:26.587251 21965 catalog_manager.cc:4841] TS b02a1d2b233b44daae6ed472429a3921 (127.27.40.131:34913): tablet 421b16fc42504de7b8b048158cf500cf (table test-table [id=80eac886cb3b461e8bc03b0e48dbb0a3]) successfully deleted
I20260809 03:36:26.594609 22681 ts_tablet_manager.cc:1905] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:36:26.596644 22681 ts_tablet_manager.cc:1918] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:36:26.596741 22681 log.cc:1196] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/421b16fc42504de7b8b048158cf500cf
I20260809 03:36:26.597050 22681 ts_tablet_manager.cc:1939] T 421b16fc42504de7b8b048158cf500cf P 65257b0b1ce34839be1bddc8098cf6af: Deleting consensus metadata
W20260809 03:36:26.599983 21967 connection.cc:535] client connection to 127.27.40.129:38701 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 03:36:26.600103 21967 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 03:36:26.600427 21961 catalog_manager.cc:4841] TS 65257b0b1ce34839be1bddc8098cf6af (127.27.40.130:42439): tablet 421b16fc42504de7b8b048158cf500cf (table test-table [id=80eac886cb3b461e8bc03b0e48dbb0a3]) successfully deleted
I20260809 03:36:26.600445 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 22260
W20260809 03:36:26.600562 21961 catalog_manager.cc:4808] TS 65257b0b1ce34839be1bddc8098cf6af (127.27.40.130:42439): delete failed for tablet 421b16fc42504de7b8b048158cf500cf because the tablet was not found. No further retry: Not found: Tablet not found: 421b16fc42504de7b8b048158cf500cf
I20260809 03:36:26.617807 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.617380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:38701 (local address 127.0.0.1:49398)
0809 03:36:26.617515 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:26.617736 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:38701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 03:36:26.618042 21967 catalog_manager.cc:4584] TS ffca517d6ce84a00b4438485d429e894 (127.27.40.129:38701): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 421b16fc42504de7b8b048158cf500cf: Network error: Client connection negotiation failed: client connection to 127.27.40.129:38701: connect: Connection refused (error 111)
I20260809 03:36:26.622344 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 22434
I20260809 03:36:26.631268 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 21942
2026-08-09T03:36:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2353 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T03:36:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:26Z Disabled control of system clock
I20260809 03:36:26.665228 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:39775
--webserver_interface=127.27.40.190
--webserver_port=0
--builtin_ntp_servers=127.27.40.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:39775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:36:26.748173 22688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:26.748397 22688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:26.748418 22688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:26.749634 22688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:26.749680 22688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:26.749693 22688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:26.749706 22688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:26.749718 22688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:26.751132 22688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:39775
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:39775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:26.751400 22688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:26.751706 22688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:26.756647 22703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:26.756690 22698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:26.756737 22688 server_base.cc:1034] running on GCE node
W20260809 03:36:26.756700 22699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:26.757193 22688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:26.757519 22688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:26.758733 22688 hybrid_clock.cc:648] HybridClock initialized: now 1786246586758722 us; error 45 us; skew 500 ppm
I20260809 03:36:26.760085 22688 webserver.cc:460] Webserver started at http://127.27.40.190:39307/ using document root <none> and password file <none>
I20260809 03:36:26.760349 22688 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:26.760397 22688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:26.760545 22688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:26.761713 22688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "482fe2d30a2246bda84a4ec589bfb37d"
format_stamp: "Formatted at 2026-08-09 03:36:26 on dist-test-slave-gkw7"
server_key: "34cebebc733dee9da148f1404c07adb5"
server_key_iv: "d0fd491282f7d0d4fa2295578f8d7d29"
server_key_version: "encryptionkey@0"
I20260809 03:36:26.762085 22688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "482fe2d30a2246bda84a4ec589bfb37d"
format_stamp: "Formatted at 2026-08-09 03:36:26 on dist-test-slave-gkw7"
server_key: "34cebebc733dee9da148f1404c07adb5"
server_key_iv: "d0fd491282f7d0d4fa2295578f8d7d29"
server_key_version: "encryptionkey@0"
I20260809 03:36:26.763391 22688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 03:36:26.764170 22712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:26.764361 22688 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:36:26.764451 22688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "482fe2d30a2246bda84a4ec589bfb37d"
format_stamp: "Formatted at 2026-08-09 03:36:26 on dist-test-slave-gkw7"
server_key: "34cebebc733dee9da148f1404c07adb5"
server_key_iv: "d0fd491282f7d0d4fa2295578f8d7d29"
server_key_version: "encryptionkey@0"
I20260809 03:36:26.764520 22688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:26.780853 22688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:26.781184 22688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:26.781323 22688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:26.785346 22688 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:39775
I20260809 03:36:26.785394 22782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:39775 every 8 connection(s)
I20260809 03:36:26.785804 22688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:26.786954 22785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:26.789652 22785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d: Bootstrap starting.
I20260809 03:36:26.789916 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 22688
I20260809 03:36:26.790066 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:26.790326 27810 external_mini_cluster.cc:1428] Setting key 1ee494965917c4b78b62db6a662d879f
I20260809 03:36:26.790518 22785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:26.790927 22785 log.cc:824] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:26.791896 22785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d: No bootstrap required, opened a new log
I20260809 03:36:26.794464 22785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482fe2d30a2246bda84a4ec589bfb37d" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 39775 } }
I20260809 03:36:26.794622 22785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:26.794651 22785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482fe2d30a2246bda84a4ec589bfb37d, State: Initialized, Role: FOLLOWER
I20260809 03:36:26.794870 22785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482fe2d30a2246bda84a4ec589bfb37d" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 39775 } }
I20260809 03:36:26.794999 22785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:26.795044 22785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:26.795153 22785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:26.795997 22785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482fe2d30a2246bda84a4ec589bfb37d" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 39775 } }
I20260809 03:36:26.796164 22785 leader_election.cc:302] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 482fe2d30a2246bda84a4ec589bfb37d; no voters:
I20260809 03:36:26.796375 22785 leader_election.cc:288] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:26.796409 22792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:26.796377 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.790887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.27.40.190:39775)
0809 03:36:26.791094 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:26.791102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:26.791128 (+ 26us) server_negotiation.cc:407] Connection header received
0809 03:36:26.791517 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:26.791522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:26.791576 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:26.791689 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:26.792023 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.792912 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.794198 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.794606 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.794753 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.795539 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:26.795544 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:26.795544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:26.795777 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:26.795825 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:26.795861 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:26.795953 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:26.795997 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 03:36:26.796101 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260809 03:36:26.796582 22792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [term 1 LEADER]: Becoming Leader. State: Replica: 482fe2d30a2246bda84a4ec589bfb37d, State: Running, Role: LEADER
I20260809 03:36:26.796676 22792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482fe2d30a2246bda84a4ec589bfb37d" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 39775 } }
I20260809 03:36:26.796718 22785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:26.797022 22792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "482fe2d30a2246bda84a4ec589bfb37d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482fe2d30a2246bda84a4ec589bfb37d" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 39775 } } }
I20260809 03:36:26.797200 22792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:26.797401 22792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 482fe2d30a2246bda84a4ec589bfb37d. Latest consensus state: current_term: 1 leader_uuid: "482fe2d30a2246bda84a4ec589bfb37d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482fe2d30a2246bda84a4ec589bfb37d" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 39775 } } }
I20260809 03:36:26.797477 22792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d [sys.catalog]: This master's current role is: LEADER
W20260809 03:36:26.798695 22815 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:36:26.798756 22815 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:36:26.798844 22798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:26.799135 22798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:26.800695 22798 catalog_manager.cc:1347] Generated new cluster ID: b0ea791ba84d40db9acde950597f7812
I20260809 03:36:26.800743 22798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:26.805994 22798 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:26.806522 22798 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:26.816145 22798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d: Generated new TSK 0
I20260809 03:36:26.816334 22798 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:26.819530 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:39775
--builtin_ntp_servers=127.27.40.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:36:26.904485 22818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:26.904703 22818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:26.904752 22818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:26.905954 22818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:26.905997 22818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:26.906023 22818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:36:26.907380 22818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:39775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:26.907641 22818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:26.907968 22818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:36:26.913034 22818 server_base.cc:1034] running on GCE node
W20260809 03:36:26.913048 22823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:26.913030 22826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:26.913273 22829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:26.913519 22818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:26.913830 22818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:26.915011 22818 hybrid_clock.cc:648] HybridClock initialized: now 1786246586914997 us; error 29 us; skew 500 ppm
I20260809 03:36:26.916672 22818 webserver.cc:460] Webserver started at http://127.27.40.129:37317/ using document root <none> and password file <none>
I20260809 03:36:26.916980 22818 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:26.917052 22818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:26.917191 22818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:26.918617 22818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "0a508c8d2c204244a4083e47091a7357"
format_stamp: "Formatted at 2026-08-09 03:36:26 on dist-test-slave-gkw7"
server_key: "39d0fc187c7029a1ce2f65dec8f7d97d"
server_key_iv: "e182c4e5eef2815934d733be71058c79"
server_key_version: "encryptionkey@0"
I20260809 03:36:26.919008 22818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "0a508c8d2c204244a4083e47091a7357"
format_stamp: "Formatted at 2026-08-09 03:36:26 on dist-test-slave-gkw7"
server_key: "39d0fc187c7029a1ce2f65dec8f7d97d"
server_key_iv: "e182c4e5eef2815934d733be71058c79"
server_key_version: "encryptionkey@0"
I20260809 03:36:26.920713 22818 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 03:36:26.921613 22837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:26.921795 22818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:36:26.921885 22818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "0a508c8d2c204244a4083e47091a7357"
format_stamp: "Formatted at 2026-08-09 03:36:26 on dist-test-slave-gkw7"
server_key: "39d0fc187c7029a1ce2f65dec8f7d97d"
server_key_iv: "e182c4e5eef2815934d733be71058c79"
server_key_version: "encryptionkey@0"
I20260809 03:36:26.921941 22818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:26.937717 22818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:26.938042 22818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:26.938215 22818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:26.938457 22818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:26.938814 22818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:26.938876 22818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:26.938922 22818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:26.938947 22818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:26.944645 22818 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:37529
I20260809 03:36:26.944691 23001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:37529 every 8 connection(s)
I20260809 03:36:26.945060 22818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:36:26.945475 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 22818
I20260809 03:36:26.945665 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:36:26.945936 27810 external_mini_cluster.cc:1428] Setting key 13fad632565a038be4054ff4e2ddf357
I20260809 03:36:26.947070 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:39775
--builtin_ntp_servers=127.27.40.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 03:36:26.951211 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.945991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:47757 (local address 127.27.40.190:39775)
0809 03:36:26.946056 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:26.946058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:26.946402 (+ 344us) server_negotiation.cc:407] Connection header received
0809 03:36:26.947083 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:26.947083 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:26.947094 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:26.947155 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:26.948059 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.948563 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.949760 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.949978 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.950067 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.950942 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:26.950944 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:26.950944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:26.950949 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:26.950963 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:26.950967 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:26.951013 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:26.951114 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 03:36:26.951145 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 03:36:26.951483 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:26.945976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:39775 (local address 127.27.40.129:47757)
0809 03:36:26.946272 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:26.946303 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:26.946977 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:26.947232 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:26.947236 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:26.947246 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:26.947966 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:26.947973 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.948667 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.948669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:26.949667 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:26.949670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:26.950080 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:26.950081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:26.950240 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:26.950784 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:36:26.950785 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:26.950822 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:36:26.950851 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:26.951032 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:26.951037 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:26.951040 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:36:26.951109 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 03:36:26.951223 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":121,"threads_started":1}
I20260809 03:36:26.952068 23003 heartbeater.cc:347] Connected to a master server at 127.27.40.190:39775
I20260809 03:36:26.952179 23003 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:26.952396 23003 heartbeater.cc:511] Master 127.27.40.190:39775 requested a full tablet report, sending...
I20260809 03:36:26.952976 22735 ts_manager.cc:194] Registered new tserver with Master: 0a508c8d2c204244a4083e47091a7357 (127.27.40.129:37529)
I20260809 03:36:26.953696 22735 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.40.129:47757
W20260809 03:36:27.031939 23012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:27.032240 23012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:27.032277 23012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:27.033434 23012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:27.033506 23012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:27.033545 23012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:36:27.034919 23012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:39775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:27.035204 23012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:27.035511 23012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:27.040628 23021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:27.040629 23024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:27.040699 23022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:27.040879 23012 server_base.cc:1034] running on GCE node
I20260809 03:36:27.041153 23012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:27.041416 23012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:27.042618 23012 hybrid_clock.cc:648] HybridClock initialized: now 1786246587042606 us; error 48 us; skew 500 ppm
I20260809 03:36:27.043852 23012 webserver.cc:460] Webserver started at http://127.27.40.130:40293/ using document root <none> and password file <none>
I20260809 03:36:27.044143 23012 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:27.044220 23012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:27.044382 23012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:27.045472 23012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "9ba942ef0aad4f058cc3c046ee49ca23"
format_stamp: "Formatted at 2026-08-09 03:36:27 on dist-test-slave-gkw7"
server_key: "76a8811e7f7635847dbe8c4e741067ef"
server_key_iv: "54dce0225423c05dff420f2fdc279663"
server_key_version: "encryptionkey@0"
I20260809 03:36:27.045919 23012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "9ba942ef0aad4f058cc3c046ee49ca23"
format_stamp: "Formatted at 2026-08-09 03:36:27 on dist-test-slave-gkw7"
server_key: "76a8811e7f7635847dbe8c4e741067ef"
server_key_iv: "54dce0225423c05dff420f2fdc279663"
server_key_version: "encryptionkey@0"
I20260809 03:36:27.047278 23012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:27.047992 23033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:27.048171 23012 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:36:27.048281 23012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "9ba942ef0aad4f058cc3c046ee49ca23"
format_stamp: "Formatted at 2026-08-09 03:36:27 on dist-test-slave-gkw7"
server_key: "76a8811e7f7635847dbe8c4e741067ef"
server_key_iv: "54dce0225423c05dff420f2fdc279663"
server_key_version: "encryptionkey@0"
I20260809 03:36:27.048357 23012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:27.066694 23012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:27.067037 23012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:27.067205 23012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:27.067492 23012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:27.067864 23012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:27.067934 23012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:27.067978 23012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:27.068005 23012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:27.074034 23012 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:45637
I20260809 03:36:27.074082 23196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:45637 every 8 connection(s)
I20260809 03:36:27.074436 23012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:36:27.079967 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.075191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:49465 (local address 127.27.40.190:39775)
0809 03:36:27.075242 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:27.075249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:27.075504 (+ 255us) server_negotiation.cc:407] Connection header received
0809 03:36:27.075994 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:27.075995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:27.076009 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:27.076122 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:27.076912 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.077476 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.078587 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.078782 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.078849 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:27.079748 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:27.079749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:27.079749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:27.079754 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:27.079768 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:27.079772 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:27.079817 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:27.079881 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 03:36:27.079905 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 03:36:27.080139 23199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.075125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:39775 (local address 127.27.40.130:49465)
0809 03:36:27.075335 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:27.075377 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:27.075877 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:27.076255 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:27.076258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:27.076267 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:27.076794 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:27.076798 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.077667 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.077669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:27.078483 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:27.078485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.078871 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.078872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:27.078986 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:27.079606 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:36:27.079607 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:27.079637 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:36:27.079661 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:27.079817 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:27.079820 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:27.079822 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 03:36:27.079870 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 03:36:27.079947 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":104,"threads_started":1}
I20260809 03:36:27.080892 23197 heartbeater.cc:347] Connected to a master server at 127.27.40.190:39775
I20260809 03:36:27.080979 23197 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:27.081210 23197 heartbeater.cc:511] Master 127.27.40.190:39775 requested a full tablet report, sending...
I20260809 03:36:27.081700 22735 ts_manager.cc:194] Registered new tserver with Master: 9ba942ef0aad4f058cc3c046ee49ca23 (127.27.40.130:45637)
I20260809 03:36:27.082044 22735 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.40.130:49465
I20260809 03:36:27.083959 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 23012
I20260809 03:36:27.084054 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:36:27.084225 27810 external_mini_cluster.cc:1428] Setting key 5c82ab34555c1fae5794a6645e3a4dc5
I20260809 03:36:27.085187 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:39775
--builtin_ntp_servers=127.27.40.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:36:27.170423 23201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:27.170636 23201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:27.170658 23201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:27.171799 23201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:27.171840 23201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:27.171865 23201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:36:27.173288 23201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:39775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:27.173557 23201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:27.173867 23201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:27.179033 23211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:27.179041 23216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:27.179137 23201 server_base.cc:1034] running on GCE node
W20260809 03:36:27.179041 23213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:27.179419 23201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:27.179617 23201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:27.180783 23201 hybrid_clock.cc:648] HybridClock initialized: now 1786246587180777 us; error 35 us; skew 500 ppm
I20260809 03:36:27.182191 23201 webserver.cc:460] Webserver started at http://127.27.40.131:34341/ using document root <none> and password file <none>
I20260809 03:36:27.182442 23201 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:27.182516 23201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:27.182677 23201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:27.183732 23201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "39934ab9cc504c6fb3c126cf8f9e5df2"
format_stamp: "Formatted at 2026-08-09 03:36:27 on dist-test-slave-gkw7"
server_key: "a79ee934942209b8abcac6c25932b845"
server_key_iv: "e5071daa55b43bc2c007c134922e425c"
server_key_version: "encryptionkey@0"
I20260809 03:36:27.184099 23201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "39934ab9cc504c6fb3c126cf8f9e5df2"
format_stamp: "Formatted at 2026-08-09 03:36:27 on dist-test-slave-gkw7"
server_key: "a79ee934942209b8abcac6c25932b845"
server_key_iv: "e5071daa55b43bc2c007c134922e425c"
server_key_version: "encryptionkey@0"
I20260809 03:36:27.185449 23201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:36:27.186292 23222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:27.186475 23201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:27.186578 23201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "39934ab9cc504c6fb3c126cf8f9e5df2"
format_stamp: "Formatted at 2026-08-09 03:36:27 on dist-test-slave-gkw7"
server_key: "a79ee934942209b8abcac6c25932b845"
server_key_iv: "e5071daa55b43bc2c007c134922e425c"
server_key_version: "encryptionkey@0"
I20260809 03:36:27.186661 23201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:27.218917 23201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:27.219348 23201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:27.219533 23201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:27.219830 23201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:27.220187 23201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:27.220247 23201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:27.220292 23201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:27.220320 23201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:27.226356 23201 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:34161
I20260809 03:36:27.226438 23392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:34161 every 8 connection(s)
I20260809 03:36:27.226801 23201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:36:27.231146 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 23201
I20260809 03:36:27.231251 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:36:27.231495 27810 external_mini_cluster.cc:1428] Setting key 8db4c31ebe08239281e0ece87318926f
I20260809 03:36:27.232396 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.227680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:48003 (local address 127.27.40.190:39775)
0809 03:36:27.227737 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:27.227739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:27.227940 (+ 201us) server_negotiation.cc:407] Connection header received
0809 03:36:27.228420 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:27.228421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:27.228432 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:27.228501 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:27.229171 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.229739 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.230752 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.231012 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.231090 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:27.232078 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:27.232080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:27.232080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:27.232085 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:27.232102 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:27.232106 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:27.232144 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:27.232287 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 03:36:27.232315 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 03:36:27.232626 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.227579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:39775 (local address 127.27.40.131:48003)
0809 03:36:27.227799 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:27.227841 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:27.228327 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:27.228557 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:27.228560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:27.228568 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:27.229066 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:27.229070 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.229859 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.229860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:27.230662 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:27.230664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.231113 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.231115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:27.231276 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:27.231899 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:36:27.231899 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:27.231938 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:36:27.231964 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:27.232188 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:27.232194 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:27.232197 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:36:27.232287 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 03:36:27.232390 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":124,"threads_started":1}
I20260809 03:36:27.233089 23395 heartbeater.cc:347] Connected to a master server at 127.27.40.190:39775
I20260809 03:36:27.233181 23395 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:27.233322 23395 heartbeater.cc:511] Master 127.27.40.190:39775 requested a full tablet report, sending...
I20260809 03:36:27.233704 22735 ts_manager.cc:194] Registered new tserver with Master: 39934ab9cc504c6fb3c126cf8f9e5df2 (127.27.40.131:34161)
I20260809 03:36:27.234086 22735 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.40.131:48003
I20260809 03:36:27.243573 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:36:27.247967 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.245017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43426 (local address 127.27.40.190:39775)
0809 03:36:27.245099 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:27.245102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:27.245248 (+ 146us) server_negotiation.cc:407] Connection header received
0809 03:36:27.245286 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:27.245287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:27.245296 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:27.245347 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:27.245488 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.246009 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.246735 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.246914 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.246967 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:27.247644 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:27.247646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:27.247646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:27.247650 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:27.247664 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:27.247668 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:27.247752 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:27.247819 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 03:36:27.247860 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:36:27.249665 22735 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:36:27.251318 22735 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:36:27.258075 23409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.253669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:34161 (local address 127.0.0.1:56910)
0809 03:36:27.253810 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:27.253863 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:27.253935 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:27.254479 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:27.254483 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:27.254485 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:27.254556 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:27.254559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.255571 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.255572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:27.256792 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:27.256794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.257763 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.257764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:27.257842 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:27.257855 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 03:36:27.257935 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 03:36:27.257975 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260809 03:36:27.258101 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.253874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56910 (local address 127.27.40.131:34161)
0809 03:36:27.254328 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:27.254332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:27.254341 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:36:27.254378 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:27.254380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:27.254393 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:27.254473 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:27.254649 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.255481 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.256891 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.257666 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.257745 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:27.257840 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:27.257967 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 03:36:27.258017 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":118,"threads_started":1}
I20260809 03:36:27.258329 23410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.253694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:45637 (local address 127.0.0.1:47366)
0809 03:36:27.253961 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:27.253973 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:27.254037 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:27.254575 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:27.254576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:27.254577 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:27.254613 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:27.254615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.255949 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.255950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:27.257096 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:27.257099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.258047 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.258048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:27.258110 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:27.258115 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:36:27.258201 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 03:36:27.258251 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":46,"thread_start_us":112,"threads_started":1}
I20260809 03:36:27.258334 23418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.254096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47366 (local address 127.27.40.130:45637)
0809 03:36:27.254354 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:27.254359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:27.254369 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:36:27.254411 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:27.254413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:27.254428 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:27.254505 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:27.254681 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.255860 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.257181 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.257960 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.258034 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:27.258126 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:27.258216 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 03:36:27.258259 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":97,"threads_started":1}
I20260809 03:36:27.258996 23412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.253704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:37529 (local address 127.0.0.1:58180)
0809 03:36:27.254126 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:27.254135 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:27.254171 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:27.254594 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:27.254595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:27.254598 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:27.254641 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:27.254643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.255961 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.255962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:27.256949 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:27.256952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.258762 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.258763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:27.258826 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:27.258830 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:36:27.258886 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 03:36:27.258925 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":33,"thread_start_us":103,"threads_started":1}
I20260809 03:36:27.259114 23108 tablet_service.cc:1467] Processing CreateTablet for tablet 7119efb4a61d4bc59f017fe23073145f (DEFAULT_TABLE table=test-table [id=d0d94a9813e84395aa3c3a667164c18f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:27.259052 23291 tablet_service.cc:1467] Processing CreateTablet for tablet 7119efb4a61d4bc59f017fe23073145f (DEFAULT_TABLE table=test-table [id=d0d94a9813e84395aa3c3a667164c18f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:27.259358 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.253955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58180 (local address 127.27.40.129:37529)
0809 03:36:27.254196 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:27.254201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:27.254212 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:36:27.254257 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:27.254259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:27.254277 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:27.254363 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:27.254753 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.255857 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.257037 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.257673 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.257768 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:27.257858 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:27.259226 (+ 1368us) server_negotiation.cc:299] Negotiation successful
0809 03:36:27.259277 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":126,"threads_started":1}
I20260809 03:36:27.259483 23108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7119efb4a61d4bc59f017fe23073145f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:27.259485 23291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7119efb4a61d4bc59f017fe23073145f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:27.260128 22903 tablet_service.cc:1467] Processing CreateTablet for tablet 7119efb4a61d4bc59f017fe23073145f (DEFAULT_TABLE table=test-table [id=d0d94a9813e84395aa3c3a667164c18f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:27.260475 22903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7119efb4a61d4bc59f017fe23073145f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:27.262418 23419 tablet_bootstrap.cc:492] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2: Bootstrap starting.
I20260809 03:36:27.263029 23420 tablet_bootstrap.cc:492] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23: Bootstrap starting.
I20260809 03:36:27.263145 23419 tablet_bootstrap.cc:654] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:27.263497 23419 log.cc:824] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:27.263759 23420 tablet_bootstrap.cc:654] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:27.264125 23420 log.cc:824] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:27.264187 23422 tablet_bootstrap.cc:492] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357: Bootstrap starting.
I20260809 03:36:27.264341 23419 tablet_bootstrap.cc:492] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2: No bootstrap required, opened a new log
I20260809 03:36:27.264436 23419 ts_tablet_manager.cc:1397] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:36:27.264945 23422 tablet_bootstrap.cc:654] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:27.265043 23420 tablet_bootstrap.cc:492] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23: No bootstrap required, opened a new log
I20260809 03:36:27.265148 23420 ts_tablet_manager.cc:1397] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:36:27.265327 23422 log.cc:824] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:27.266172 23422 tablet_bootstrap.cc:492] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357: No bootstrap required, opened a new log
I20260809 03:36:27.266247 23422 ts_tablet_manager.cc:1397] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:36:27.266808 23419 raft_consensus.cc:348] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39934ab9cc504c6fb3c126cf8f9e5df2" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34161 } } peers { permanent_uuid: "0a508c8d2c204244a4083e47091a7357" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37529 } } peers { permanent_uuid: "9ba942ef0aad4f058cc3c046ee49ca23" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45637 } }
I20260809 03:36:27.267025 23419 raft_consensus.cc:374] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:27.267099 23419 raft_consensus.cc:729] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39934ab9cc504c6fb3c126cf8f9e5df2, State: Initialized, Role: FOLLOWER
I20260809 03:36:27.267275 23419 consensus_queue.cc:260] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39934ab9cc504c6fb3c126cf8f9e5df2" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34161 } } peers { permanent_uuid: "0a508c8d2c204244a4083e47091a7357" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37529 } } peers { permanent_uuid: "9ba942ef0aad4f058cc3c046ee49ca23" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45637 } }
I20260809 03:36:27.267689 23419 ts_tablet_manager.cc:1428] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:36:27.267820 23395 heartbeater.cc:503] Master 127.27.40.190:39775 was elected leader, sending a full tablet report...
I20260809 03:36:27.267831 23420 raft_consensus.cc:348] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39934ab9cc504c6fb3c126cf8f9e5df2" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34161 } } peers { permanent_uuid: "0a508c8d2c204244a4083e47091a7357" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37529 } } peers { permanent_uuid: "9ba942ef0aad4f058cc3c046ee49ca23" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45637 } }
I20260809 03:36:27.267983 23420 raft_consensus.cc:374] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:27.268070 23420 raft_consensus.cc:729] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ba942ef0aad4f058cc3c046ee49ca23, State: Initialized, Role: FOLLOWER
I20260809 03:36:27.268213 23420 consensus_queue.cc:260] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39934ab9cc504c6fb3c126cf8f9e5df2" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34161 } } peers { permanent_uuid: "0a508c8d2c204244a4083e47091a7357" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37529 } } peers { permanent_uuid: "9ba942ef0aad4f058cc3c046ee49ca23" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45637 } }
I20260809 03:36:27.268468 23420 ts_tablet_manager.cc:1428] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 03:36:27.268503 23197 heartbeater.cc:503] Master 127.27.40.190:39775 was elected leader, sending a full tablet report...
I20260809 03:36:27.268906 23422 raft_consensus.cc:348] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39934ab9cc504c6fb3c126cf8f9e5df2" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34161 } } peers { permanent_uuid: "0a508c8d2c204244a4083e47091a7357" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37529 } } peers { permanent_uuid: "9ba942ef0aad4f058cc3c046ee49ca23" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45637 } }
I20260809 03:36:27.269088 23422 raft_consensus.cc:374] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:27.269169 23422 raft_consensus.cc:729] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a508c8d2c204244a4083e47091a7357, State: Initialized, Role: FOLLOWER
I20260809 03:36:27.269313 23422 consensus_queue.cc:260] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39934ab9cc504c6fb3c126cf8f9e5df2" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34161 } } peers { permanent_uuid: "0a508c8d2c204244a4083e47091a7357" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37529 } } peers { permanent_uuid: "9ba942ef0aad4f058cc3c046ee49ca23" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45637 } }
I20260809 03:36:27.269626 23422 ts_tablet_manager.cc:1428] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:36:27.269690 23003 heartbeater.cc:503] Master 127.27.40.190:39775 was elected leader, sending a full tablet report...
I20260809 03:36:27.301666 23431 raft_consensus.cc:482] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:36:27.301801 23431 raft_consensus.cc:504] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39934ab9cc504c6fb3c126cf8f9e5df2" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34161 } } peers { permanent_uuid: "0a508c8d2c204244a4083e47091a7357" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37529 } } peers { permanent_uuid: "9ba942ef0aad4f058cc3c046ee49ca23" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45637 } }
I20260809 03:36:27.302125 23431 leader_election.cc:288] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39934ab9cc504c6fb3c126cf8f9e5df2 (127.27.40.131:34161), 9ba942ef0aad4f058cc3c046ee49ca23 (127.27.40.130:45637)
I20260809 03:36:27.306181 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.302489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:44855 (local address 127.27.40.131:34161)
0809 03:36:27.302543 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:27.302545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:27.302552 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:36:27.302583 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:27.302584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:27.302593 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:27.302651 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:27.302875 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.303631 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.305213 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.305788 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.305878 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:27.305886 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:27.306062 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 03:36:27.306105 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 03:36:27.306180 23418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.302378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:55787 (local address 127.27.40.130:45637)
0809 03:36:27.302449 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:27.302452 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:27.302516 (+ 64us) server_negotiation.cc:407] Connection header received
0809 03:36:27.302586 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:27.302587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:27.302600 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:27.302657 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:27.302872 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.303631 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.305208 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.305792 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.305877 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:27.305886 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:27.306063 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 03:36:27.306105 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:36:27.306183 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.302289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:34161 (local address 127.27.40.129:44855)
0809 03:36:27.302358 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:27.302376 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:27.302439 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:27.302677 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:27.302678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:27.302680 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:27.302791 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:27.302793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.303745 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.303746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:27.305115 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:27.305118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.305884 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.305885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:27.305980 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:27.305985 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:36:27.306045 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 03:36:27.306098 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":106}
I20260809 03:36:27.306190 23433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.302319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:45637 (local address 127.27.40.129:55787)
0809 03:36:27.302458 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:27.302469 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:27.302534 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:27.302677 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:27.302678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:27.302680 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:27.302794 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:27.302796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.303745 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.303746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:27.305129 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:27.305131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.305886 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.305887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:27.305986 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:27.305989 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:36:27.306045 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 03:36:27.306096 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":86,"thread_start_us":88,"threads_started":1}
I20260809 03:36:27.306643 23322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7119efb4a61d4bc59f017fe23073145f" candidate_uuid: "0a508c8d2c204244a4083e47091a7357" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39934ab9cc504c6fb3c126cf8f9e5df2" is_pre_election: true
I20260809 03:36:27.306641 23139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7119efb4a61d4bc59f017fe23073145f" candidate_uuid: "0a508c8d2c204244a4083e47091a7357" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ba942ef0aad4f058cc3c046ee49ca23" is_pre_election: true
I20260809 03:36:27.306818 23322 raft_consensus.cc:2454] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a508c8d2c204244a4083e47091a7357 in term 0.
I20260809 03:36:27.306830 23139 raft_consensus.cc:2454] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a508c8d2c204244a4083e47091a7357 in term 0.
I20260809 03:36:27.307096 22845 leader_election.cc:302] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a508c8d2c204244a4083e47091a7357, 9ba942ef0aad4f058cc3c046ee49ca23; no voters:
I20260809 03:36:27.307243 23431 raft_consensus.cc:2781] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:36:27.307303 23431 raft_consensus.cc:482] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:36:27.307356 23431 raft_consensus.cc:3037] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:27.308074 23431 raft_consensus.cc:504] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39934ab9cc504c6fb3c126cf8f9e5df2" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34161 } } peers { permanent_uuid: "0a508c8d2c204244a4083e47091a7357" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37529 } } peers { permanent_uuid: "9ba942ef0aad4f058cc3c046ee49ca23" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45637 } }
I20260809 03:36:27.308244 23431 leader_election.cc:288] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [CANDIDATE]: Term 1 election: Requested vote from peers 39934ab9cc504c6fb3c126cf8f9e5df2 (127.27.40.131:34161), 9ba942ef0aad4f058cc3c046ee49ca23 (127.27.40.130:45637)
I20260809 03:36:27.308418 23139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7119efb4a61d4bc59f017fe23073145f" candidate_uuid: "0a508c8d2c204244a4083e47091a7357" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ba942ef0aad4f058cc3c046ee49ca23"
I20260809 03:36:27.308413 23322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7119efb4a61d4bc59f017fe23073145f" candidate_uuid: "0a508c8d2c204244a4083e47091a7357" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39934ab9cc504c6fb3c126cf8f9e5df2"
I20260809 03:36:27.308491 23322 raft_consensus.cc:3037] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:27.308491 23139 raft_consensus.cc:3037] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:27.309396 23322 raft_consensus.cc:2454] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a508c8d2c204244a4083e47091a7357 in term 1.
I20260809 03:36:27.309396 23139 raft_consensus.cc:2454] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a508c8d2c204244a4083e47091a7357 in term 1.
I20260809 03:36:27.309573 22845 leader_election.cc:302] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a508c8d2c204244a4083e47091a7357, 9ba942ef0aad4f058cc3c046ee49ca23; no voters:
I20260809 03:36:27.309682 23431 raft_consensus.cc:2781] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:27.309830 23431 raft_consensus.cc:686] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [term 1 LEADER]: Becoming Leader. State: Replica: 0a508c8d2c204244a4083e47091a7357, State: Running, Role: LEADER
I20260809 03:36:27.309954 23431 consensus_queue.cc:237] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39934ab9cc504c6fb3c126cf8f9e5df2" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34161 } } peers { permanent_uuid: "0a508c8d2c204244a4083e47091a7357" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37529 } } peers { permanent_uuid: "9ba942ef0aad4f058cc3c046ee49ca23" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45637 } }
I20260809 03:36:27.310793 22735 catalog_manager.cc:5495] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a508c8d2c204244a4083e47091a7357 (127.27.40.129). New cstate: current_term: 1 leader_uuid: "0a508c8d2c204244a4083e47091a7357" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39934ab9cc504c6fb3c126cf8f9e5df2" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a508c8d2c204244a4083e47091a7357" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 37529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ba942ef0aad4f058cc3c046ee49ca23" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45637 } health_report { overall_health: UNKNOWN } } }
I20260809 03:36:27.317700 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.315295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58182 (local address 127.27.40.129:37529)
0809 03:36:27.315339 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:27.315341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:27.315351 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:36:27.315429 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:27.315430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:27.315449 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:27.315534 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:36:27.315674 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.316129 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.316884 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:27.317083 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:27.317160 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:27.317312 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:36:27.317409 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:36:27.317518 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:27.317583 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 03:36:27.317594 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 03:36:27.319492 23322 raft_consensus.cc:1261] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2 [term 1 FOLLOWER]: Refusing update from remote peer 0a508c8d2c204244a4083e47091a7357: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:36:27.319525 23139 raft_consensus.cc:1261] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23 [term 1 FOLLOWER]: Refusing update from remote peer 0a508c8d2c204244a4083e47091a7357: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:36:27.319837 23431 consensus_queue.cc:1035] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39934ab9cc504c6fb3c126cf8f9e5df2" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 34161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:36:27.319996 23435 consensus_queue.cc:1035] T 7119efb4a61d4bc59f017fe23073145f P 0a508c8d2c204244a4083e47091a7357 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ba942ef0aad4f058cc3c046ee49ca23" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:36:27.322566 23439 mvcc.cc:205] Tried to move back new op lower bound from 7316466021658333184 to 7316466021622145024. Current Snapshot: MvccSnapshot[applied={T|T < 7316466021658333184}]
I20260809 03:36:27.322741 23442 mvcc.cc:205] Tried to move back new op lower bound from 7316466021658333184 to 7316466021622145024. Current Snapshot: MvccSnapshot[applied={T|T < 7316466021658333184}]
I20260809 03:36:27.322878 23440 mvcc.cc:205] Tried to move back new op lower bound from 7316466021658333184 to 7316466021622145024. Current Snapshot: MvccSnapshot[applied={T|T < 7316466021658333184}]
I20260809 03:36:27.325007 22733 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:36:27.325101 22733 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:36:27.325995 22733 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 482fe2d30a2246bda84a4ec589bfb37d: Sending DeleteTablet for 3 replicas of tablet 7119efb4a61d4bc59f017fe23073145f
I20260809 03:36:27.326310 23291 tablet_service.cc:1514] Processing DeleteTablet for tablet 7119efb4a61d4bc59f017fe23073145f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:27 UTC) from {username='slave'} at 127.0.0.1:56910
I20260809 03:36:27.326371 23108 tablet_service.cc:1514] Processing DeleteTablet for tablet 7119efb4a61d4bc59f017fe23073145f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:27 UTC) from {username='slave'} at 127.0.0.1:47366
I20260809 03:36:27.326413 22903 tablet_service.cc:1514] Processing DeleteTablet for tablet 7119efb4a61d4bc59f017fe23073145f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:27 UTC) from {username='slave'} at 127.0.0.1:58180
I20260809 03:36:27.326629 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 22818
I20260809 03:36:27.326653 23453 tablet_replica.cc:330] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23: stopping tablet replica
I20260809 03:36:27.326752 23453 raft_consensus.cc:2229] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:36:27.326854 23453 raft_consensus.cc:2258] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:27.333698 23453 ts_tablet_manager.cc:1905] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:36:27.336064 23453 ts_tablet_manager.cc:1918] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:36:27.336135 23453 log.cc:1196] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/7119efb4a61d4bc59f017fe23073145f
I20260809 03:36:27.336346 23453 ts_tablet_manager.cc:1939] T 7119efb4a61d4bc59f017fe23073145f P 9ba942ef0aad4f058cc3c046ee49ca23: Deleting consensus metadata
I20260809 03:36:27.337205 22716 catalog_manager.cc:4841] TS 9ba942ef0aad4f058cc3c046ee49ca23 (127.27.40.130:45637): tablet 7119efb4a61d4bc59f017fe23073145f (table test-table [id=d0d94a9813e84395aa3c3a667164c18f]) successfully deleted
I20260809 03:36:27.338755 23452 tablet_replica.cc:330] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2: stopping tablet replica
I20260809 03:36:27.338919 23452 raft_consensus.cc:2229] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:36:27.339058 23452 raft_consensus.cc:2258] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:27.339540 23452 ts_tablet_manager.cc:1905] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:36:27.340876 23452 ts_tablet_manager.cc:1918] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:36:27.340939 23452 log.cc:1196] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/7119efb4a61d4bc59f017fe23073145f
I20260809 03:36:27.341180 23452 ts_tablet_manager.cc:1939] T 7119efb4a61d4bc59f017fe23073145f P 39934ab9cc504c6fb3c126cf8f9e5df2: Deleting consensus metadata
I20260809 03:36:27.341624 22714 catalog_manager.cc:4841] TS 39934ab9cc504c6fb3c126cf8f9e5df2 (127.27.40.131:34161): tablet 7119efb4a61d4bc59f017fe23073145f (table test-table [id=d0d94a9813e84395aa3c3a667164c18f]) successfully deleted
W20260809 03:36:27.343335 22715 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.40.129:37529 (error 108)
I20260809 03:36:27.343679 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 23012
I20260809 03:36:27.347882 23412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:27.343766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:37529 (local address 127.0.0.1:58190)
0809 03:36:27.347771 (+ 4005us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:27.347845 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:37529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3975}
W20260809 03:36:27.350558 22715 catalog_manager.cc:4584] TS 0a508c8d2c204244a4083e47091a7357 (127.27.40.129:37529): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7119efb4a61d4bc59f017fe23073145f: Network error: Client connection negotiation failed: client connection to 127.27.40.129:37529: connect: Connection refused (error 111)
I20260809 03:36:27.364722 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 23201
I20260809 03:36:27.376305 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 22688
2026-08-09T03:36:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (747 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:27 dist-test-slave-gkw7 krb5kdc[23459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:27 dist-test-slave-gkw7 krb5kdc[23459](info): set up 2 sockets
Aug 09 03:36:27 dist-test-slave-gkw7 krb5kdc[23459](info): commencing operation
krb5kdc: starting...
W20260809 03:36:28.602571 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:28 dist-test-slave-gkw7 krb5kdc[23459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:36:28.733155 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:35303
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:35303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:28.818446 23478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:28.818738 23478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:28.818785 23478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:28.820089 23478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:28.820169 23478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:28.820197 23478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:28.820222 23478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:28.820254 23478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:28.821928 23478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:35303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:35303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:28.822319 23478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:28.822592 23478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:28.828038 23494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:28.828099 23488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:28.828035 23489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:28.828370 23478 server_base.cc:1034] running on GCE node
I20260809 03:36:28.828634 23478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:28.828990 23478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:28.830137 23478 hybrid_clock.cc:648] HybridClock initialized: now 1786246588830114 us; error 46 us; skew 500 ppm
Aug 09 03:36:28 dist-test-slave-gkw7 krb5kdc[23459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246588, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:28.832629 23478 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:36:28.833004 23478 webserver.cc:460] Webserver started at http://127.27.40.190:42855/ using document root <none> and password file <none>
I20260809 03:36:28.833197 23478 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:28.833238 23478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:28.833324 23478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:28.834389 23478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "499df123082c4f35a6e376b43a8756bb"
format_stamp: "Formatted at 2026-08-09 03:36:28 on dist-test-slave-gkw7"
server_key: "cc55f6c89eb6f6d2d827415a0e50062d"
server_key_iv: "6d77d2990c5eb3823f0439728d8751ee"
server_key_version: "encryptionkey@0"
I20260809 03:36:28.834702 23478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "499df123082c4f35a6e376b43a8756bb"
format_stamp: "Formatted at 2026-08-09 03:36:28 on dist-test-slave-gkw7"
server_key: "cc55f6c89eb6f6d2d827415a0e50062d"
server_key_iv: "6d77d2990c5eb3823f0439728d8751ee"
server_key_version: "encryptionkey@0"
I20260809 03:36:28.835994 23478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 03:36:28.836719 23511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:28.836916 23478 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:36:28.837028 23478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "499df123082c4f35a6e376b43a8756bb"
format_stamp: "Formatted at 2026-08-09 03:36:28 on dist-test-slave-gkw7"
server_key: "cc55f6c89eb6f6d2d827415a0e50062d"
server_key_iv: "6d77d2990c5eb3823f0439728d8751ee"
server_key_version: "encryptionkey@0"
I20260809 03:36:28.837090 23478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:28.852038 23478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:28.853929 23478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:28.854099 23478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:28.858132 23478 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:35303
I20260809 03:36:28.858183 23578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:35303 every 8 connection(s)
I20260809 03:36:28.858578 23478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:28.858875 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 23478
I20260809 03:36:28.859021 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:28.859277 27810 external_mini_cluster.cc:1428] Setting key e67fdce2b49cdcf8f20d6b70247a2c07
I20260809 03:36:28.859705 23579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:28.863093 23579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb: Bootstrap starting.
I20260809 03:36:28.863787 23579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:28.864121 23579 log.cc:824] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:28.864892 23579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb: No bootstrap required, opened a new log
Aug 09 03:36:28 dist-test-slave-gkw7 krb5kdc[23459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:28.867266 23579 raft_consensus.cc:348] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499df123082c4f35a6e376b43a8756bb" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35303 } }
I20260809 03:36:28.867425 23579 raft_consensus.cc:374] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:28.867475 23579 raft_consensus.cc:729] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499df123082c4f35a6e376b43a8756bb, State: Initialized, Role: FOLLOWER
I20260809 03:36:28.867656 23579 consensus_queue.cc:260] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499df123082c4f35a6e376b43a8756bb" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35303 } }
I20260809 03:36:28.867756 23579 raft_consensus.cc:388] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:28.867802 23579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:28.867851 23579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:28.868561 23579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499df123082c4f35a6e376b43a8756bb" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35303 } }
I20260809 03:36:28.868685 23579 leader_election.cc:302] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 499df123082c4f35a6e376b43a8756bb; no voters:
I20260809 03:36:28.868860 23579 leader_election.cc:288] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:28.868933 23586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:28.869108 23586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [term 1 LEADER]: Becoming Leader. State: Replica: 499df123082c4f35a6e376b43a8756bb, State: Running, Role: LEADER
I20260809 03:36:28.869130 23579 sys_catalog.cc:564] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:28.869212 23586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499df123082c4f35a6e376b43a8756bb" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35303 } }
I20260809 03:36:28.869511 23586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 499df123082c4f35a6e376b43a8756bb. Latest consensus state: current_term: 1 leader_uuid: "499df123082c4f35a6e376b43a8756bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499df123082c4f35a6e376b43a8756bb" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35303 } } }
I20260809 03:36:28.869609 23586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:28.869711 23588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "499df123082c4f35a6e376b43a8756bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499df123082c4f35a6e376b43a8756bb" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35303 } } }
I20260809 03:36:28.869786 23588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:28.869901 23600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:28.870415 23600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:28.871248 23584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:28.859845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53668 (local address 127.27.40.190:35303)
0809 03:36:28.860030 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:28.860039 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:28.860073 (+ 34us) server_negotiation.cc:407] Connection header received
0809 03:36:28.860496 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:28.860502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:28.860546 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:28.860664 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:28.861478 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:28.862478 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:28.863590 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:28.864038 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:28.864131 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:28.866914 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:28.866918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:28.866919 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:28.866929 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:28.868995 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:28.869451 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:28.869454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:28.869454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:28.869588 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:28.870045 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:28.870046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:28.870047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:28.870416 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:28.870625 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:28.870919 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 03:36:28.871064 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":110,"threads_started":1}
I20260809 03:36:28.872143 23600 catalog_manager.cc:1347] Generated new cluster ID: da427dce90334791a5e41de47e384851
I20260809 03:36:28.872196 23600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:28.890210 23600 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:28.890769 23600 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:28.896551 23600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb: Generated new TSK 0
I20260809 03:36:28.896759 23600 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.40.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
Entry for principal kudu/127.27.40.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab.
I20260809 03:36:28.932289 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:35303
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:29.016523 23619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:29.016770 23619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:29.016812 23619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:29.018023 23619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:29.018095 23619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:29.018142 23619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:36:29.019687 23619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.129.keytab
--principal=kudu/127.27.40.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:35303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:29.019955 23619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:29.020197 23619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:29.025622 23627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:29.025722 23629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:29.025795 23619 server_base.cc:1034] running on GCE node
W20260809 03:36:29.025630 23626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:29.026058 23619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:29.026314 23619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:29.027457 23619 hybrid_clock.cc:648] HybridClock initialized: now 1786246589027444 us; error 28 us; skew 500 ppm
Aug 09 03:36:29 dist-test-slave-gkw7 krb5kdc[23459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246589, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:29.030113 23619 init.cc:377] Logged in from keytab as kudu/127.27.40.129@KRBTEST.COM (short username kudu)
I20260809 03:36:29.030552 23619 webserver.cc:460] Webserver started at http://127.27.40.129:37439/ using document root <none> and password file <none>
I20260809 03:36:29.030817 23619 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:29.030889 23619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:29.031038 23619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:29.032104 23619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "2c62ecc266044394b16851727ab54a13"
format_stamp: "Formatted at 2026-08-09 03:36:29 on dist-test-slave-gkw7"
server_key: "89fe8b34dd940df2c5e6d4415d3ef6d9"
server_key_iv: "378ebf67defc94cdb5aecaef9eb44371"
server_key_version: "encryptionkey@0"
I20260809 03:36:29.032486 23619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "2c62ecc266044394b16851727ab54a13"
format_stamp: "Formatted at 2026-08-09 03:36:29 on dist-test-slave-gkw7"
server_key: "89fe8b34dd940df2c5e6d4415d3ef6d9"
server_key_iv: "378ebf67defc94cdb5aecaef9eb44371"
server_key_version: "encryptionkey@0"
I20260809 03:36:29.033955 23619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:36:29.034714 23640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:29.034896 23619 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:36:29.034989 23619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "2c62ecc266044394b16851727ab54a13"
format_stamp: "Formatted at 2026-08-09 03:36:29 on dist-test-slave-gkw7"
server_key: "89fe8b34dd940df2c5e6d4415d3ef6d9"
server_key_iv: "378ebf67defc94cdb5aecaef9eb44371"
server_key_version: "encryptionkey@0"
I20260809 03:36:29.035072 23619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:29.044186 23619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:29.046541 23619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:29.046736 23619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:29.047014 23619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:29.047360 23619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:29.047421 23619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:29.047468 23619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:29.047495 23619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:29.054349 23619 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:44849
I20260809 03:36:29.054404 23808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:44849 every 8 connection(s)
I20260809 03:36:29.054776 23619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:36:29.058240 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 23619
I20260809 03:36:29.058370 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:36:29.058602 27810 external_mini_cluster.cc:1428] Setting key a3d4a11ef7be27d8efccfe6b7714dcf3
Aug 09 03:36:29 dist-test-slave-gkw7 krb5kdc[23459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246589, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.129@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:29.066160 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.055590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:35303 (local address 127.27.40.129:32965)
0809 03:36:29.055831 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:29.055871 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:29.056383 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:29.056622 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:29.056626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:29.057222 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:29.057420 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:29.057425 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.058244 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.058245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:29.059475 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:29.059479 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.059882 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.059883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:29.060030 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.060575 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:29.060585 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:29.062653 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:29.064652 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:29.064664 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:29.064666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:29.064888 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:29.065112 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:29.065113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:29.065113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:29.065244 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:29.065627 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:29.065635 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:29.065769 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 03:36:29.065859 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 03:36:29.065987 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":121,"threads_started":1}
I20260809 03:36:29.066162 23584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.055617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:32965 (local address 127.27.40.190:35303)
0809 03:36:29.055693 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:29.055696 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:29.055964 (+ 268us) server_negotiation.cc:407] Connection header received
0809 03:36:29.056504 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:29.056505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:29.056517 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:29.056566 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:29.057570 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.058141 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.059582 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.059789 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.059854 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.062748 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:29.062750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:29.062750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:29.062758 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:29.064534 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:29.064969 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:29.064970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:29.064971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:29.065041 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:29.065315 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:29.065316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:29.065316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:29.065483 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:29.065625 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:29.065910 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 03:36:29.066018 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 03:36:29.066759 23809 heartbeater.cc:347] Connected to a master server at 127.27.40.190:35303
I20260809 03:36:29.066857 23809 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:29.067054 23809 heartbeater.cc:511] Master 127.27.40.190:35303 requested a full tablet report, sending...
I20260809 03:36:29.067613 23531 ts_manager.cc:194] Registered new tserver with Master: 2c62ecc266044394b16851727ab54a13 (127.27.40.129:44849)
I20260809 03:36:29.068181 23531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.129@KRBTEST.COM'} at 127.27.40.129:32965
WARNING: no policy specified for kudu/127.27.40.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
Entry for principal kudu/127.27.40.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab.
I20260809 03:36:29.089247 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:35303
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:29.176957 23816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:29.177222 23816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:29.177268 23816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:29.178596 23816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:29.178684 23816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:29.178737 23816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:36:29.180377 23816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.130.keytab
--principal=kudu/127.27.40.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:35303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:29.180665 23816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:29.180917 23816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:29.186436 23824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:29.186421 23827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:29.186411 23823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:29.186794 23816 server_base.cc:1034] running on GCE node
I20260809 03:36:29.187027 23816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:29.187300 23816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:29.188460 23816 hybrid_clock.cc:648] HybridClock initialized: now 1786246589188453 us; error 40 us; skew 500 ppm
Aug 09 03:36:29 dist-test-slave-gkw7 krb5kdc[23459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246589, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:29.190989 23816 init.cc:377] Logged in from keytab as kudu/127.27.40.130@KRBTEST.COM (short username kudu)
I20260809 03:36:29.191479 23816 webserver.cc:460] Webserver started at http://127.27.40.130:38653/ using document root <none> and password file <none>
I20260809 03:36:29.191704 23816 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:29.191771 23816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:29.191922 23816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:29.192961 23816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c"
format_stamp: "Formatted at 2026-08-09 03:36:29 on dist-test-slave-gkw7"
server_key: "2460055e1939d0ecf7205f4b780888ab"
server_key_iv: "472ac1753c22a2d9f30ed735857dc05e"
server_key_version: "encryptionkey@0"
I20260809 03:36:29.193357 23816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c"
format_stamp: "Formatted at 2026-08-09 03:36:29 on dist-test-slave-gkw7"
server_key: "2460055e1939d0ecf7205f4b780888ab"
server_key_iv: "472ac1753c22a2d9f30ed735857dc05e"
server_key_version: "encryptionkey@0"
I20260809 03:36:29.194805 23816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:36:29.195570 23838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:29.195745 23816 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:36:29.195843 23816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c"
format_stamp: "Formatted at 2026-08-09 03:36:29 on dist-test-slave-gkw7"
server_key: "2460055e1939d0ecf7205f4b780888ab"
server_key_iv: "472ac1753c22a2d9f30ed735857dc05e"
server_key_version: "encryptionkey@0"
I20260809 03:36:29.195925 23816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:29.210598 23816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:29.212486 23816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:29.212656 23816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:29.212911 23816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:29.213220 23816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:29.213279 23816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:29.213321 23816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:29.213361 23816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:29.219432 23816 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:37251
I20260809 03:36:29.219487 24014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:37251 every 8 connection(s)
I20260809 03:36:29.219800 23816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:36:29.225206 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 23816
I20260809 03:36:29.225332 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:36:29.225577 27810 external_mini_cluster.cc:1428] Setting key 0e4a2f743313fac6dd0a75615222a281
Aug 09 03:36:29 dist-test-slave-gkw7 krb5kdc[23459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246589, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.130@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:29.230726 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.220498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:35303 (local address 127.27.40.130:59913)
0809 03:36:29.220691 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:29.220734 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:29.221213 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:29.221426 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:29.221429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:29.222102 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:29.222301 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:29.222306 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.223021 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.223022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:29.223920 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:29.223923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.224296 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.224297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:29.224442 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.225099 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:29.225110 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:29.226981 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:29.229149 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:29.229157 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:29.229160 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:29.229514 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:29.229770 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:29.229770 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:29.229771 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:29.229917 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:29.230238 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:29.230244 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:29.230355 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 03:36:29.230430 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 03:36:29.230548 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":103,"threads_started":1}
I20260809 03:36:29.230731 23584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.220522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:59913 (local address 127.27.40.190:35303)
0809 03:36:29.220598 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:29.220600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:29.220827 (+ 227us) server_negotiation.cc:407] Connection header received
0809 03:36:29.221297 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:29.221298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:29.221308 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:29.221364 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:29.222393 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.222882 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.224001 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.224206 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.224289 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.227468 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:29.227470 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:29.227470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:29.227479 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:29.229067 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:29.229619 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:29.229620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:29.229620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:29.229691 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:29.230015 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:29.230016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:29.230016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:29.230177 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:29.230244 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:29.230490 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 03:36:29.230582 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:36:29.231249 24016 heartbeater.cc:347] Connected to a master server at 127.27.40.190:35303
I20260809 03:36:29.231345 24016 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:29.231544 24016 heartbeater.cc:511] Master 127.27.40.190:35303 requested a full tablet report, sending...
I20260809 03:36:29.231916 23531 ts_manager.cc:194] Registered new tserver with Master: f8412fe0fb0646ce98ad3abd58fe5c9c (127.27.40.130:37251)
I20260809 03:36:29.232255 23531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.130@KRBTEST.COM'} at 127.27.40.130:59913
WARNING: no policy specified for kudu/127.27.40.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
Entry for principal kudu/127.27.40.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab.
I20260809 03:36:29.254207 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:35303
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:29.342607 24026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:29.342830 24026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:29.342857 24026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:29.344007 24026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:29.344066 24026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:29.344095 24026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:36:29.345670 24026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.131.keytab
--principal=kudu/127.27.40.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:35303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:29.345939 24026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:29.346196 24026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:29.351559 24034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:29.351598 24038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:29.351560 24035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:29.351610 24026 server_base.cc:1034] running on GCE node
I20260809 03:36:29.351964 24026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:29.352255 24026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:29.353423 24026 hybrid_clock.cc:648] HybridClock initialized: now 1786246589353419 us; error 36 us; skew 500 ppm
Aug 09 03:36:29 dist-test-slave-gkw7 krb5kdc[23459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246589, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:29.355944 24026 init.cc:377] Logged in from keytab as kudu/127.27.40.131@KRBTEST.COM (short username kudu)
I20260809 03:36:29.356398 24026 webserver.cc:460] Webserver started at http://127.27.40.131:39827/ using document root <none> and password file <none>
I20260809 03:36:29.356652 24026 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:29.356722 24026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:29.356877 24026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:29.357991 24026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "90a1a12fd7924173abb9eb026c8680d0"
format_stamp: "Formatted at 2026-08-09 03:36:29 on dist-test-slave-gkw7"
server_key: "c2ee785e7b6baead61e149b7e7a16ff2"
server_key_iv: "2bb1d7c8b2d2ebd40ce8210f5233cd5e"
server_key_version: "encryptionkey@0"
I20260809 03:36:29.358366 24026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "90a1a12fd7924173abb9eb026c8680d0"
format_stamp: "Formatted at 2026-08-09 03:36:29 on dist-test-slave-gkw7"
server_key: "c2ee785e7b6baead61e149b7e7a16ff2"
server_key_iv: "2bb1d7c8b2d2ebd40ce8210f5233cd5e"
server_key_version: "encryptionkey@0"
I20260809 03:36:29.359771 24026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:36:29.360487 24049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:29.360661 24026 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:36:29.360754 24026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "90a1a12fd7924173abb9eb026c8680d0"
format_stamp: "Formatted at 2026-08-09 03:36:29 on dist-test-slave-gkw7"
server_key: "c2ee785e7b6baead61e149b7e7a16ff2"
server_key_iv: "2bb1d7c8b2d2ebd40ce8210f5233cd5e"
server_key_version: "encryptionkey@0"
I20260809 03:36:29.360836 24026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:29.370918 24026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:29.373272 24026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:29.373448 24026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:29.373713 24026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:29.374034 24026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:29.374089 24026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:29.374130 24026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:29.374167 24026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:29.380196 24026 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:40473
I20260809 03:36:29.380236 24213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:40473 every 8 connection(s)
I20260809 03:36:29.380597 24026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
Aug 09 03:36:29 dist-test-slave-gkw7 krb5kdc[23459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246589, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.131@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:29.389647 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 24026
I20260809 03:36:29.389758 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:36:29.389765 23584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.381376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:51739 (local address 127.27.40.190:35303)
0809 03:36:29.381426 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:29.381429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:29.381652 (+ 223us) server_negotiation.cc:407] Connection header received
0809 03:36:29.382230 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:29.382232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:29.382245 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:29.382319 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:29.383314 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.383826 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.384865 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.385065 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.385152 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.387277 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:29.387279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:29.387279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:29.387285 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:29.388497 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:29.388856 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:29.388857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:29.388858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:29.388909 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:29.389113 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:29.389113 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:29.389114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:29.389227 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:29.389278 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:29.389524 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 03:36:29.389650 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 03:36:29.389935 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.381323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:35303 (local address 127.27.40.131:51739)
0809 03:36:29.381476 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:29.381559 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:29.382084 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:29.382399 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:29.382404 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:29.383015 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:29.383210 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:29.383215 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.383925 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.383926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:29.384754 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:29.384756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.385164 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.385165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:29.385301 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.385838 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:29.385847 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:29.387193 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:29.388570 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:29.388576 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:29.388578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:29.388790 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:29.388966 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:29.388967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:29.388967 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:29.389060 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:29.389282 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:29.389286 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:29.389399 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 03:36:29.389478 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 03:36:29.389665 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
I20260809 03:36:29.389961 27810 external_mini_cluster.cc:1428] Setting key e8c45274514184874bcb639dcd8b45d8
I20260809 03:36:29.390456 24214 heartbeater.cc:347] Connected to a master server at 127.27.40.190:35303
I20260809 03:36:29.390559 24214 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:29.390785 24214 heartbeater.cc:511] Master 127.27.40.190:35303 requested a full tablet report, sending...
I20260809 03:36:29.391183 23531 ts_manager.cc:194] Registered new tserver with Master: 90a1a12fd7924173abb9eb026c8680d0 (127.27.40.131:40473)
I20260809 03:36:29.391521 23531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.40.131@KRBTEST.COM'} at 127.27.40.131:51739
I20260809 03:36:29.401919 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:36:29.409348 23584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.403290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53684 (local address 127.27.40.190:35303)
0809 03:36:29.403359 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:29.403362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:29.403434 (+ 72us) server_negotiation.cc:407] Connection header received
0809 03:36:29.403515 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:29.403516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:29.403526 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:29.403583 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:29.404145 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.404683 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.405868 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.406064 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.406117 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.407237 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:29.407238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:29.407238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:29.407245 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:29.408408 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:29.408713 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:29.408713 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:29.408714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:29.408766 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:29.408927 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:29.408928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:29.408928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:29.409061 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:29.409136 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:29.409223 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 03:36:29.409282 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 03:36:29.410880 23531 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:36:29.412513 23531 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:36:29.419236 24228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.414890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:44849 (local address 127.0.0.1:50032)
0809 03:36:29.415059 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:29.415100 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:29.415194 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:29.415578 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:29.415581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:29.415583 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:29.415650 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:29.415652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.417059 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.417060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:29.418141 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:29.418143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.418900 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.418901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:29.418995 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.419009 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 03:36:29.419100 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 03:36:29.419145 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":63,"threads_started":1}
I20260809 03:36:29.419250 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.415098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50032 (local address 127.27.40.129:44849)
0809 03:36:29.415404 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:29.415409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:29.415420 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:36:29.415465 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:29.415468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:29.415485 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:29.415558 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:29.415745 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.416952 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.418235 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.418798 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.418887 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.418948 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:29.419122 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 03:36:29.419166 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":86,"threads_started":1}
I20260809 03:36:29.419415 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.415270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41812 (local address 127.27.40.131:40473)
0809 03:36:29.415469 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:29.415472 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:29.415611 (+ 139us) server_negotiation.cc:407] Connection header received
0809 03:36:29.415656 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:29.415670 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:29.415687 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:29.415761 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:29.415963 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.416632 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.418092 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.418806 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.418881 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.418942 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:29.419155 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 03:36:29.419200 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":102,"threads_started":1}
I20260809 03:36:29.419447 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.414996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42950 (local address 127.27.40.130:37251)
0809 03:36:29.415361 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:29.415367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:29.415378 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:36:29.415419 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:29.415421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:29.415758 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:29.415841 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:29.415997 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.417051 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.418130 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.418806 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.419192 (+ 386us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.419267 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:29.419294 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 03:36:29.419344 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":86,"threads_started":1}
I20260809 03:36:29.419466 24229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.414890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:37251 (local address 127.0.0.1:42950)
0809 03:36:29.415190 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:29.415202 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:29.415268 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:29.415850 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:29.415851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:29.415853 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:29.415904 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:29.415906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.417138 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.417139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:29.418003 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:29.418006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.418911 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.418912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:29.418996 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.419014 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 03:36:29.419056 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 03:36:29.419110 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":8,"thread_start_us":86,"threads_started":1}
I20260809 03:36:29.419631 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.415090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:40473 (local address 127.0.0.1:41812)
0809 03:36:29.415493 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:29.415504 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:29.415552 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:29.415843 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:29.415844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:29.415846 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:29.415902 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:29.415904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.416735 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.416736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:29.417999 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:29.418002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.418971 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.418972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:29.419022 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.419025 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:36:29.419073 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 03:36:29.419110 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":3,"thread_start_us":64,"threads_started":1}
I20260809 03:36:29.420375 23707 tablet_service.cc:1467] Processing CreateTablet for tablet e2248dba233c43d09fad6e8d2d18d41e (DEFAULT_TABLE table=test-table [id=226de611417847fa9e4c7f2cdbf70666]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:29.420429 23914 tablet_service.cc:1467] Processing CreateTablet for tablet e2248dba233c43d09fad6e8d2d18d41e (DEFAULT_TABLE table=test-table [id=226de611417847fa9e4c7f2cdbf70666]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:29.420428 24112 tablet_service.cc:1467] Processing CreateTablet for tablet e2248dba233c43d09fad6e8d2d18d41e (DEFAULT_TABLE table=test-table [id=226de611417847fa9e4c7f2cdbf70666]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:29.420783 24112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2248dba233c43d09fad6e8d2d18d41e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:29.420783 23914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2248dba233c43d09fad6e8d2d18d41e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:29.420792 23707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2248dba233c43d09fad6e8d2d18d41e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:29.423955 24237 tablet_bootstrap.cc:492] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13: Bootstrap starting.
I20260809 03:36:29.424289 24240 tablet_bootstrap.cc:492] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c: Bootstrap starting.
I20260809 03:36:29.424686 24237 tablet_bootstrap.cc:654] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:29.424902 24240 tablet_bootstrap.cc:654] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:29.425211 24237 log.cc:824] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:29.425431 24240 log.cc:824] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:29.425833 24238 tablet_bootstrap.cc:492] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0: Bootstrap starting.
I20260809 03:36:29.426177 24237 tablet_bootstrap.cc:492] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13: No bootstrap required, opened a new log
I20260809 03:36:29.426203 24240 tablet_bootstrap.cc:492] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c: No bootstrap required, opened a new log
I20260809 03:36:29.426255 24237 ts_tablet_manager.cc:1397] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:36:29.426285 24240 ts_tablet_manager.cc:1397] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 03:36:29.426517 24238 tablet_bootstrap.cc:654] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:29.427080 24238 log.cc:824] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:29.427891 24238 tablet_bootstrap.cc:492] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0: No bootstrap required, opened a new log
I20260809 03:36:29.427960 24238 ts_tablet_manager.cc:1397] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:36:29.428205 24240 raft_consensus.cc:348] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 37251 } } peers { permanent_uuid: "90a1a12fd7924173abb9eb026c8680d0" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 40473 } } peers { permanent_uuid: "2c62ecc266044394b16851727ab54a13" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44849 } }
I20260809 03:36:29.428376 24240 raft_consensus.cc:374] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:29.428422 24240 raft_consensus.cc:729] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8412fe0fb0646ce98ad3abd58fe5c9c, State: Initialized, Role: FOLLOWER
I20260809 03:36:29.428380 24237 raft_consensus.cc:348] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 37251 } } peers { permanent_uuid: "90a1a12fd7924173abb9eb026c8680d0" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 40473 } } peers { permanent_uuid: "2c62ecc266044394b16851727ab54a13" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44849 } }
I20260809 03:36:29.428536 24240 consensus_queue.cc:260] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 37251 } } peers { permanent_uuid: "90a1a12fd7924173abb9eb026c8680d0" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 40473 } } peers { permanent_uuid: "2c62ecc266044394b16851727ab54a13" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44849 } }
I20260809 03:36:29.428582 24237 raft_consensus.cc:374] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:29.428630 24237 raft_consensus.cc:729] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c62ecc266044394b16851727ab54a13, State: Initialized, Role: FOLLOWER
I20260809 03:36:29.428822 24240 ts_tablet_manager.cc:1428] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 03:36:29.428787 24237 consensus_queue.cc:260] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 37251 } } peers { permanent_uuid: "90a1a12fd7924173abb9eb026c8680d0" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 40473 } } peers { permanent_uuid: "2c62ecc266044394b16851727ab54a13" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44849 } }
I20260809 03:36:29.428848 24016 heartbeater.cc:503] Master 127.27.40.190:35303 was elected leader, sending a full tablet report...
I20260809 03:36:29.429061 24237 ts_tablet_manager.cc:1428] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:36:29.429104 23809 heartbeater.cc:503] Master 127.27.40.190:35303 was elected leader, sending a full tablet report...
I20260809 03:36:29.430022 24238 raft_consensus.cc:348] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 37251 } } peers { permanent_uuid: "90a1a12fd7924173abb9eb026c8680d0" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 40473 } } peers { permanent_uuid: "2c62ecc266044394b16851727ab54a13" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44849 } }
I20260809 03:36:29.430193 24238 raft_consensus.cc:374] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:29.430272 24238 raft_consensus.cc:729] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a1a12fd7924173abb9eb026c8680d0, State: Initialized, Role: FOLLOWER
I20260809 03:36:29.430392 24238 consensus_queue.cc:260] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 37251 } } peers { permanent_uuid: "90a1a12fd7924173abb9eb026c8680d0" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 40473 } } peers { permanent_uuid: "2c62ecc266044394b16851727ab54a13" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44849 } }
I20260809 03:36:29.430675 24238 ts_tablet_manager.cc:1428] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:36:29.430691 24214 heartbeater.cc:503] Master 127.27.40.190:35303 was elected leader, sending a full tablet report...
W20260809 03:36:29.470335 24019 tablet.cc:2369] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:36:29.555647 23810 tablet.cc:2369] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:36:29.604444 24254 raft_consensus.cc:482] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:36:29.604583 24254 raft_consensus.cc:504] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 37251 } } peers { permanent_uuid: "90a1a12fd7924173abb9eb026c8680d0" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 40473 } } peers { permanent_uuid: "2c62ecc266044394b16851727ab54a13" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44849 } }
I20260809 03:36:29.604979 24254 leader_election.cc:288] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90a1a12fd7924173abb9eb026c8680d0 (127.27.40.131:40473), 2c62ecc266044394b16851727ab54a13 (127.27.40.129:44849)
I20260809 03:36:29.609398 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.605321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:38471 (local address 127.27.40.131:40473)
0809 03:36:29.605372 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:29.605374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:29.605386 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:36:29.605453 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:29.605456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:29.605470 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:29.605583 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:29.605804 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.606573 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.608372 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.608938 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.609017 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.609026 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:29.609276 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 03:36:29.609317 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 03:36:29.609398 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.605363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:34543 (local address 127.27.40.129:44849)
0809 03:36:29.605418 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:29.605420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:29.605551 (+ 131us) server_negotiation.cc:407] Connection header received
0809 03:36:29.605675 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:29.605676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:29.605684 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:29.605731 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:29.605907 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.606581 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.608375 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.608938 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.609017 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.609027 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:29.609275 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 03:36:29.609317 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 03:36:29.609483 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.605246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:40473 (local address 127.27.40.130:38471)
0809 03:36:29.605311 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:29.605325 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:29.605376 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:29.605594 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:29.605596 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:29.605597 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:29.605688 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:29.605692 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.606678 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.606679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:29.608271 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:29.608274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.609080 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.609081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:29.609172 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.609175 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:36:29.609273 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 03:36:29.609328 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":118}
I20260809 03:36:29.609496 24258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.605246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:44849 (local address 127.27.40.130:34543)
0809 03:36:29.605457 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:29.605467 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:29.605593 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:29.605763 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:29.605764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:29.605766 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:29.605838 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:29.605839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.606678 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.606679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:29.608271 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:29.608274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.609062 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.609064 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:29.609169 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.609175 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:36:29.609268 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 03:36:29.609328 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":86,"thread_start_us":129,"threads_started":1}
I20260809 03:36:29.610010 23733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2248dba233c43d09fad6e8d2d18d41e" candidate_uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c62ecc266044394b16851727ab54a13" is_pre_election: true
I20260809 03:36:29.610020 24141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2248dba233c43d09fad6e8d2d18d41e" candidate_uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a1a12fd7924173abb9eb026c8680d0" is_pre_election: true
I20260809 03:36:29.610183 23733 raft_consensus.cc:2454] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8412fe0fb0646ce98ad3abd58fe5c9c in term 0.
I20260809 03:36:29.610201 24141 raft_consensus.cc:2454] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8412fe0fb0646ce98ad3abd58fe5c9c in term 0.
I20260809 03:36:29.610489 23843 leader_election.cc:302] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c62ecc266044394b16851727ab54a13, f8412fe0fb0646ce98ad3abd58fe5c9c; no voters:
I20260809 03:36:29.610648 24254 raft_consensus.cc:2781] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:36:29.610707 24254 raft_consensus.cc:482] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:36:29.610736 24254 raft_consensus.cc:3037] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:29.611470 24254 raft_consensus.cc:504] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 37251 } } peers { permanent_uuid: "90a1a12fd7924173abb9eb026c8680d0" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 40473 } } peers { permanent_uuid: "2c62ecc266044394b16851727ab54a13" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44849 } }
I20260809 03:36:29.611610 24254 leader_election.cc:288] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [CANDIDATE]: Term 1 election: Requested vote from peers 90a1a12fd7924173abb9eb026c8680d0 (127.27.40.131:40473), 2c62ecc266044394b16851727ab54a13 (127.27.40.129:44849)
I20260809 03:36:29.611819 23733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2248dba233c43d09fad6e8d2d18d41e" candidate_uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c62ecc266044394b16851727ab54a13"
I20260809 03:36:29.611853 24141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2248dba233c43d09fad6e8d2d18d41e" candidate_uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a1a12fd7924173abb9eb026c8680d0"
I20260809 03:36:29.611927 23733 raft_consensus.cc:3037] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:29.611948 24141 raft_consensus.cc:3037] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:29.612668 24141 raft_consensus.cc:2454] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8412fe0fb0646ce98ad3abd58fe5c9c in term 1.
I20260809 03:36:29.612849 23733 raft_consensus.cc:2454] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8412fe0fb0646ce98ad3abd58fe5c9c in term 1.
I20260809 03:36:29.612952 23845 leader_election.cc:302] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90a1a12fd7924173abb9eb026c8680d0, f8412fe0fb0646ce98ad3abd58fe5c9c; no voters:
I20260809 03:36:29.613058 24254 raft_consensus.cc:2781] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:29.613252 24254 raft_consensus.cc:686] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [term 1 LEADER]: Becoming Leader. State: Replica: f8412fe0fb0646ce98ad3abd58fe5c9c, State: Running, Role: LEADER
I20260809 03:36:29.613368 24254 consensus_queue.cc:237] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 37251 } } peers { permanent_uuid: "90a1a12fd7924173abb9eb026c8680d0" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 40473 } } peers { permanent_uuid: "2c62ecc266044394b16851727ab54a13" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44849 } }
I20260809 03:36:29.614328 23530 catalog_manager.cc:5495] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c reported cstate change: term changed from 0 to 1, leader changed from <none> to f8412fe0fb0646ce98ad3abd58fe5c9c (127.27.40.130). New cstate: current_term: 1 leader_uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8412fe0fb0646ce98ad3abd58fe5c9c" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 37251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90a1a12fd7924173abb9eb026c8680d0" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c62ecc266044394b16851727ab54a13" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44849 } health_report { overall_health: UNKNOWN } } }
W20260809 03:36:29.631254 24215 tablet.cc:2369] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:36:29.647099 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.644138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42960 (local address 127.27.40.130:37251)
0809 03:36:29.644216 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:29.644218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:29.644230 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:36:29.644358 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:29.644359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:29.644382 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:29.644472 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:36:29.644635 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.645181 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.646168 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.646387 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.646477 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.646684 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:36:29.646795 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:36:29.646898 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:29.646957 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 03:36:29.646985 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:36:29.649164 24141 raft_consensus.cc:1261] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0 [term 1 FOLLOWER]: Refusing update from remote peer f8412fe0fb0646ce98ad3abd58fe5c9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:36:29.649200 23733 raft_consensus.cc:1261] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13 [term 1 FOLLOWER]: Refusing update from remote peer f8412fe0fb0646ce98ad3abd58fe5c9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:36:29.649632 24259 consensus_queue.cc:1035] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "90a1a12fd7924173abb9eb026c8680d0" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:36:29.649784 24254 consensus_queue.cc:1035] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c62ecc266044394b16851727ab54a13" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 44849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:36:29.652124 24264 mvcc.cc:205] Tried to move back new op lower bound from 7316466031200043008 to 7316466031057121280. Current Snapshot: MvccSnapshot[applied={T|T < 7316466031200043008}]
I20260809 03:36:29.652418 24265 mvcc.cc:205] Tried to move back new op lower bound from 7316466031200043008 to 7316466031057121280. Current Snapshot: MvccSnapshot[applied={T|T < 7316466031200043008}]
I20260809 03:36:29.654923 23531 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:36:29.655038 23531 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:36:29.656037 23531 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 499df123082c4f35a6e376b43a8756bb: Sending DeleteTablet for 3 replicas of tablet e2248dba233c43d09fad6e8d2d18d41e
I20260809 03:36:29.656453 23914 tablet_service.cc:1514] Processing DeleteTablet for tablet e2248dba233c43d09fad6e8d2d18d41e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:29 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:42950
I20260809 03:36:29.656502 24112 tablet_service.cc:1514] Processing DeleteTablet for tablet e2248dba233c43d09fad6e8d2d18d41e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:29 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:41812
I20260809 03:36:29.656635 23707 tablet_service.cc:1514] Processing DeleteTablet for tablet e2248dba233c43d09fad6e8d2d18d41e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:29 UTC) from {username='kudu', principal='kudu/127.27.40.190@KRBTEST.COM'} at 127.0.0.1:50032
I20260809 03:36:29.656862 24279 tablet_replica.cc:330] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13: stopping tablet replica
I20260809 03:36:29.656984 24279 raft_consensus.cc:2229] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:36:29.657014 24278 tablet_replica.cc:330] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0: stopping tablet replica
I20260809 03:36:29.657120 24279 raft_consensus.cc:2258] T e2248dba233c43d09fad6e8d2d18d41e P 2c62ecc266044394b16851727ab54a13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:29.657140 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 23619
I20260809 03:36:29.665728 24278 raft_consensus.cc:2229] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:36:29.665922 24278 raft_consensus.cc:2258] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:29.666234 24276 tablet_replica.cc:330] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c: stopping tablet replica
I20260809 03:36:29.666369 24276 raft_consensus.cc:2229] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:36:29.666525 24276 raft_consensus.cc:2258] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:29.666843 24278 ts_tablet_manager.cc:1905] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:36:29.667066 24276 ts_tablet_manager.cc:1905] T e2248dba233c43d09fad6e8d2d18d41e P f8412fe0fb0646ce98ad3abd58fe5c9c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 03:36:29.667999 23514 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.40.129:44849 (error 108)
I20260809 03:36:29.668298 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 23816
I20260809 03:36:29.681994 24278 ts_tablet_manager.cc:1918] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:36:29.682109 24278 log.cc:1196] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/e2248dba233c43d09fad6e8d2d18d41e
I20260809 03:36:29.682411 24278 ts_tablet_manager.cc:1939] T e2248dba233c43d09fad6e8d2d18d41e P 90a1a12fd7924173abb9eb026c8680d0: Deleting consensus metadata
I20260809 03:36:29.683017 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.682815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:44849 (local address 127.0.0.1:50034)
0809 03:36:29.682922 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:29.682973 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:44849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260809 03:36:29.683275 23514 catalog_manager.cc:4584] TS 2c62ecc266044394b16851727ab54a13 (127.27.40.129:44849): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e2248dba233c43d09fad6e8d2d18d41e: Network error: Client connection negotiation failed: client connection to 127.27.40.129:44849: connect: Connection refused (error 111)
I20260809 03:36:29.683020 23515 catalog_manager.cc:4841] TS 90a1a12fd7924173abb9eb026c8680d0 (127.27.40.131:40473): tablet e2248dba233c43d09fad6e8d2d18d41e (table test-table [id=226de611417847fa9e4c7f2cdbf70666]) successfully deleted
I20260809 03:36:29.692338 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 24026
I20260809 03:36:29.692444 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.692289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:37251 (local address 127.0.0.1:42964)
0809 03:36:29.692362 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:29.692399 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.130:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 03:36:29.703435 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 23478
2026-08-09T03:36:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2324 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T03:36:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:29Z Disabled control of system clock
I20260809 03:36:29.738114 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:37197
--webserver_interface=127.27.40.190
--webserver_port=0
--builtin_ntp_servers=127.27.40.148:35567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:37197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:36:29.820173 24290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:29.820408 24290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:29.820429 24290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:29.821666 24290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:29.821707 24290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:29.821720 24290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:29.821733 24290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:29.821745 24290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:29.823210 24290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:35567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:37197
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:37197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:29.823469 24290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:29.823791 24290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:29.829071 24306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:29.829080 24309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:29.829139 24301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:29.829149 24290 server_base.cc:1034] running on GCE node
I20260809 03:36:29.829579 24290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:29.829860 24290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:29.831007 24290 hybrid_clock.cc:648] HybridClock initialized: now 1786246589830998 us; error 43 us; skew 500 ppm
I20260809 03:36:29.832366 24290 webserver.cc:460] Webserver started at http://127.27.40.190:45461/ using document root <none> and password file <none>
I20260809 03:36:29.832636 24290 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:29.832682 24290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:29.832857 24290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:29.834064 24290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "7ecac47575804012a1dbcb866acb0358"
format_stamp: "Formatted at 2026-08-09 03:36:29 on dist-test-slave-gkw7"
server_key: "6bea0473b05941bf52a5356ccc03dd0a"
server_key_iv: "98c6afa2b5b12b874892d0ffb63e7bb3"
server_key_version: "encryptionkey@0"
I20260809 03:36:29.834465 24290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "7ecac47575804012a1dbcb866acb0358"
format_stamp: "Formatted at 2026-08-09 03:36:29 on dist-test-slave-gkw7"
server_key: "6bea0473b05941bf52a5356ccc03dd0a"
server_key_iv: "98c6afa2b5b12b874892d0ffb63e7bb3"
server_key_version: "encryptionkey@0"
I20260809 03:36:29.835817 24290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:36:29.836584 24317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:29.836786 24290 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:29.836884 24290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "7ecac47575804012a1dbcb866acb0358"
format_stamp: "Formatted at 2026-08-09 03:36:29 on dist-test-slave-gkw7"
server_key: "6bea0473b05941bf52a5356ccc03dd0a"
server_key_iv: "98c6afa2b5b12b874892d0ffb63e7bb3"
server_key_version: "encryptionkey@0"
I20260809 03:36:29.836941 24290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:29.867447 24290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:29.867780 24290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:29.867897 24290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:29.872526 24290 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:37197
I20260809 03:36:29.872570 24385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:37197 every 8 connection(s)
I20260809 03:36:29.872957 24290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:29.874053 24387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:29.876462 24387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358: Bootstrap starting.
I20260809 03:36:29.877076 24387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:29.877409 24387 log.cc:824] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:29.878168 24387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358: No bootstrap required, opened a new log
I20260809 03:36:29.879803 24387 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecac47575804012a1dbcb866acb0358" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37197 } }
I20260809 03:36:29.879958 24387 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:29.879980 24387 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecac47575804012a1dbcb866acb0358, State: Initialized, Role: FOLLOWER
I20260809 03:36:29.880156 24387 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecac47575804012a1dbcb866acb0358" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37197 } }
I20260809 03:36:29.880223 24387 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:29.880264 24387 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:29.880316 24387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:29.880877 24387 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecac47575804012a1dbcb866acb0358" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37197 } }
I20260809 03:36:29.880992 24387 leader_election.cc:302] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ecac47575804012a1dbcb866acb0358; no voters:
I20260809 03:36:29.881202 24387 leader_election.cc:288] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:29.881264 24391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:29.881464 24391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [term 1 LEADER]: Becoming Leader. State: Replica: 7ecac47575804012a1dbcb866acb0358, State: Running, Role: LEADER
I20260809 03:36:29.881485 24387 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:29.881585 24391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecac47575804012a1dbcb866acb0358" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37197 } }
I20260809 03:36:29.882004 24393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ecac47575804012a1dbcb866acb0358. Latest consensus state: current_term: 1 leader_uuid: "7ecac47575804012a1dbcb866acb0358" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecac47575804012a1dbcb866acb0358" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37197 } } }
I20260809 03:36:29.882099 24392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ecac47575804012a1dbcb866acb0358" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecac47575804012a1dbcb866acb0358" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 37197 } } }
I20260809 03:36:29.882149 24393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:29.882154 24392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:29.882439 24405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:29.883085 24405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:29.883184 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 24290
I20260809 03:36:29.883301 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:29.883502 27810 external_mini_cluster.cc:1428] Setting key 41c02e599a736b95788f1f46e629f720
I20260809 03:36:29.884997 24405 catalog_manager.cc:1347] Generated new cluster ID: d80d7f71e5ea44ff8fc891b56c4e3396
I20260809 03:36:29.885054 24405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:29.890271 24422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:29.883967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58636 (local address 127.27.40.190:37197)
0809 03:36:29.884317 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:29.884325 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:29.884357 (+ 32us) server_negotiation.cc:407] Connection header received
0809 03:36:29.884493 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:29.884500 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:29.884603 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:29.884764 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:29.885075 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.886719 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.887827 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:29.888536 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:29.888688 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:29.889428 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:29.889433 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:29.889433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:29.889732 (+ 299us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:29.889781 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:29.889798 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:29.889875 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:29.889942 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 03:36:29.890028 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":75,"threads_started":1}
I20260809 03:36:29.897644 24405 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:29.898137 24405 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:29.900610 24405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358: Generated new TSK 0
I20260809 03:36:29.900810 24405 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:29.906741 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.129:0
--local_ip_for_outbound_sockets=127.27.40.129
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37197
--builtin_ntp_servers=127.27.40.148:35567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:36:29.990553 24425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:29.990757 24425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:29.990779 24425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:29.991938 24425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:29.991981 24425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:29.992007 24425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.129
I20260809 03:36:29.993412 24425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:35567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.40.129
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.129
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:29.993664 24425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:29.993999 24425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:29.999141 24433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:29.999203 24431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:29.999207 24435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:29.999689 24425 server_base.cc:1034] running on GCE node
I20260809 03:36:29.999886 24425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:30.000216 24425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:30.001384 24425 hybrid_clock.cc:648] HybridClock initialized: now 1786246590001362 us; error 40 us; skew 500 ppm
I20260809 03:36:30.002960 24425 webserver.cc:460] Webserver started at http://127.27.40.129:40747/ using document root <none> and password file <none>
I20260809 03:36:30.003273 24425 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:30.003326 24425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:30.003476 24425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:30.004590 24425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/data/instance:
uuid: "0419427f4cec4dcfbe131b65ae65467c"
format_stamp: "Formatted at 2026-08-09 03:36:29 on dist-test-slave-gkw7"
server_key: "16df188478afa8ad86fa23251496eecc"
server_key_iv: "a4ff86b3f13446c5358e0fda96cfa037"
server_key_version: "encryptionkey@0"
I20260809 03:36:30.004997 24425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance:
uuid: "0419427f4cec4dcfbe131b65ae65467c"
format_stamp: "Formatted at 2026-08-09 03:36:29 on dist-test-slave-gkw7"
server_key: "16df188478afa8ad86fa23251496eecc"
server_key_iv: "a4ff86b3f13446c5358e0fda96cfa037"
server_key_version: "encryptionkey@0"
I20260809 03:36:30.006536 24425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:36:30.007385 24446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:30.007588 24425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:36:30.007664 24425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/wal
uuid: "0419427f4cec4dcfbe131b65ae65467c"
format_stamp: "Formatted at 2026-08-09 03:36:29 on dist-test-slave-gkw7"
server_key: "16df188478afa8ad86fa23251496eecc"
server_key_iv: "a4ff86b3f13446c5358e0fda96cfa037"
server_key_version: "encryptionkey@0"
I20260809 03:36:30.007766 24425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:30.024611 24425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:30.025004 24425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:30.025182 24425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:30.025461 24425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:30.025882 24425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:30.025920 24425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:30.025980 24425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:30.026010 24425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:30.032943 24425 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.129:43721
I20260809 03:36:30.032994 24607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.129:43721 every 8 connection(s)
I20260809 03:36:30.033360 24425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/data/info.pb
I20260809 03:36:30.039055 24422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.034256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.129:55115 (local address 127.27.40.190:37197)
0809 03:36:30.034344 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:30.034347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:30.034567 (+ 220us) server_negotiation.cc:407] Connection header received
0809 03:36:30.035156 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:30.035158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:30.035173 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:30.035233 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:30.035960 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.036477 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.037506 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.037735 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.037824 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.038740 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:30.038741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:30.038741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:30.038748 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:30.038763 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:30.038767 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:30.038840 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:30.038944 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 03:36:30.038974 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 03:36:30.039265 24612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.034177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37197 (local address 127.27.40.129:55115)
0809 03:36:30.034437 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:30.034471 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:30.035021 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:30.035315 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:30.035319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:30.035328 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:30.035843 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:30.035847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.036576 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.036577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:30.037410 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:30.037413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.037837 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.037838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:30.037985 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.038567 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:36:30.038568 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:30.038597 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:36:30.038625 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:30.038840 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:30.038847 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:30.038850 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:36:30.038933 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 03:36:30.039037 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":106,"threads_started":1}
I20260809 03:36:30.039767 24610 heartbeater.cc:347] Connected to a master server at 127.27.40.190:37197
I20260809 03:36:30.039868 24610 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:30.040060 24610 heartbeater.cc:511] Master 127.27.40.190:37197 requested a full tablet report, sending...
I20260809 03:36:30.040728 24338 ts_manager.cc:194] Registered new tserver with Master: 0419427f4cec4dcfbe131b65ae65467c (127.27.40.129:43721)
I20260809 03:36:30.041432 24338 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.40.129:55115
I20260809 03:36:30.043068 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 24425
I20260809 03:36:30.043171 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-0/wal/instance
I20260809 03:36:30.043334 27810 external_mini_cluster.cc:1428] Setting key 3cf532ae52858287acd0090f3ebcc4e6
I20260809 03:36:30.044237 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.130:0
--local_ip_for_outbound_sockets=127.27.40.130
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37197
--builtin_ntp_servers=127.27.40.148:35567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:36:30.127183 24616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:30.127434 24616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:30.127470 24616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:30.128588 24616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:30.128652 24616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:30.128695 24616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.130
I20260809 03:36:30.130116 24616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:35567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.40.130
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.130
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:30.130378 24616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:30.130669 24616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:36:30.135908 24616 server_base.cc:1034] running on GCE node
W20260809 03:36:30.135876 24626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:30.135882 24627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:30.135890 24630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:30.136296 24616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:30.136500 24616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:30.137633 24616 hybrid_clock.cc:648] HybridClock initialized: now 1786246590137613 us; error 36 us; skew 500 ppm
I20260809 03:36:30.138994 24616 webserver.cc:460] Webserver started at http://127.27.40.130:37277/ using document root <none> and password file <none>
I20260809 03:36:30.139249 24616 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:30.139298 24616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:30.139464 24616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:30.140630 24616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/data/instance:
uuid: "7c6aacb354754bbea16edd30b57f3f3b"
format_stamp: "Formatted at 2026-08-09 03:36:30 on dist-test-slave-gkw7"
server_key: "9398959f6218abb2738c3958ef9741c5"
server_key_iv: "437f9480842734a0a890c79a9cb55f3c"
server_key_version: "encryptionkey@0"
I20260809 03:36:30.140985 24616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance:
uuid: "7c6aacb354754bbea16edd30b57f3f3b"
format_stamp: "Formatted at 2026-08-09 03:36:30 on dist-test-slave-gkw7"
server_key: "9398959f6218abb2738c3958ef9741c5"
server_key_iv: "437f9480842734a0a890c79a9cb55f3c"
server_key_version: "encryptionkey@0"
I20260809 03:36:30.142452 24616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:36:30.143253 24640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:30.143424 24616 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:36:30.143530 24616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/wal
uuid: "7c6aacb354754bbea16edd30b57f3f3b"
format_stamp: "Formatted at 2026-08-09 03:36:30 on dist-test-slave-gkw7"
server_key: "9398959f6218abb2738c3958ef9741c5"
server_key_iv: "437f9480842734a0a890c79a9cb55f3c"
server_key_version: "encryptionkey@0"
I20260809 03:36:30.143613 24616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:30.154558 24616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:30.154851 24616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:30.155006 24616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:30.155551 24616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:30.155973 24616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:30.156030 24616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:30.156078 24616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:30.156107 24616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:30.162313 24616 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.130:35095
I20260809 03:36:30.162391 24808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.130:35095 every 8 connection(s)
I20260809 03:36:30.162736 24616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/data/info.pb
I20260809 03:36:30.167969 24422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.163526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:60471 (local address 127.27.40.190:37197)
0809 03:36:30.163605 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:30.163607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:30.163842 (+ 235us) server_negotiation.cc:407] Connection header received
0809 03:36:30.164313 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:30.164314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:30.164324 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:30.164380 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:30.165031 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.165576 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.166513 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.166702 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.166756 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.167732 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:30.167733 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:30.167734 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:30.167738 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:30.167753 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:30.167757 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:30.167821 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:30.167887 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 03:36:30.167911 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 03:36:30.168236 24811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.163507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37197 (local address 127.27.40.130:60471)
0809 03:36:30.163684 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:30.163726 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:30.164207 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:30.164437 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:30.164440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:30.164448 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:30.164948 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:30.164952 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.165660 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.165660 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:30.166433 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:30.166435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.166778 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.166779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:30.166887 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.167578 (+ 691us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:36:30.167579 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:30.167616 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:36:30.167637 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:30.167823 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:30.167825 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:30.167828 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:36:30.167874 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 03:36:30.167950 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260809 03:36:30.168757 24809 heartbeater.cc:347] Connected to a master server at 127.27.40.190:37197
I20260809 03:36:30.168846 24809 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:30.169039 24809 heartbeater.cc:511] Master 127.27.40.190:37197 requested a full tablet report, sending...
I20260809 03:36:30.169468 24338 ts_manager.cc:194] Registered new tserver with Master: 7c6aacb354754bbea16edd30b57f3f3b (127.27.40.130:35095)
I20260809 03:36:30.169770 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 24616
I20260809 03:36:30.169935 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/wal/instance
I20260809 03:36:30.170017 24338 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.40.130:60471
I20260809 03:36:30.170173 27810 external_mini_cluster.cc:1428] Setting key b9b2bfb54832819859a61372c5bd6bef
I20260809 03:36:30.171020 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.40.131:0
--local_ip_for_outbound_sockets=127.27.40.131
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37197
--builtin_ntp_servers=127.27.40.148:35567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:36:30.258555 24812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:30.258769 24812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:30.258790 24812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:30.259862 24812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:30.259897 24812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:30.259922 24812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.40.131
I20260809 03:36:30.261353 24812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:35567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.40.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.40.131
--webserver_port=0
--tserver_master_addrs=127.27.40.190:37197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.40.131
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:30.261610 24812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:30.261883 24812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:30.266881 24818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:30.266911 24812 server_base.cc:1034] running on GCE node
W20260809 03:36:30.266819 24821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:30.266830 24819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:30.267238 24812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:30.267503 24812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:30.268661 24812 hybrid_clock.cc:648] HybridClock initialized: now 1786246590268638 us; error 37 us; skew 500 ppm
I20260809 03:36:30.270182 24812 webserver.cc:460] Webserver started at http://127.27.40.131:39337/ using document root <none> and password file <none>
I20260809 03:36:30.270447 24812 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:30.270505 24812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:30.270658 24812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:30.272271 24812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/data/instance:
uuid: "384580df8f014e17ad05ef5016a7cd69"
format_stamp: "Formatted at 2026-08-09 03:36:30 on dist-test-slave-gkw7"
server_key: "b74fe2bbcc0296ef19a2a7b887f1f62e"
server_key_iv: "70021255ddb2fa13af615bb09c27e3c3"
server_key_version: "encryptionkey@0"
I20260809 03:36:30.272742 24812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance:
uuid: "384580df8f014e17ad05ef5016a7cd69"
format_stamp: "Formatted at 2026-08-09 03:36:30 on dist-test-slave-gkw7"
server_key: "b74fe2bbcc0296ef19a2a7b887f1f62e"
server_key_iv: "70021255ddb2fa13af615bb09c27e3c3"
server_key_version: "encryptionkey@0"
I20260809 03:36:30.274691 24812 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 03:36:30.275578 24828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:30.275759 24812 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:30.275862 24812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/wal
uuid: "384580df8f014e17ad05ef5016a7cd69"
format_stamp: "Formatted at 2026-08-09 03:36:30 on dist-test-slave-gkw7"
server_key: "b74fe2bbcc0296ef19a2a7b887f1f62e"
server_key_iv: "70021255ddb2fa13af615bb09c27e3c3"
server_key_version: "encryptionkey@0"
I20260809 03:36:30.275951 24812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:30.292817 24812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:30.293133 24812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:30.293284 24812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:30.293591 24812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:30.294001 24812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:30.294046 24812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:30.294077 24812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:30.294098 24812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:30.300433 24812 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.131:45587
I20260809 03:36:30.300566 24996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.131:45587 every 8 connection(s)
I20260809 03:36:30.300855 24812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/data/info.pb
I20260809 03:36:30.306607 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 24812
I20260809 03:36:30.306676 24422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.301674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.131:36345 (local address 127.27.40.190:37197)
0809 03:36:30.301743 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:30.301744 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:30.302011 (+ 267us) server_negotiation.cc:407] Connection header received
0809 03:36:30.302583 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:30.302585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:30.302600 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:30.302655 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:30.303408 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.303948 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.304992 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.305217 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.305307 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.306308 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:30.306309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:30.306310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:30.306316 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:30.306333 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:30.306338 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:30.306419 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:30.306551 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 03:36:30.306588 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 03:36:30.306763 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/wal/instance
I20260809 03:36:30.306905 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.301655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:37197 (local address 127.27.40.131:36345)
0809 03:36:30.301883 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:30.301918 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:30.302459 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:30.302744 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:30.302748 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:30.302757 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:30.303298 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:30.303303 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.304065 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.304066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:30.304873 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:30.304875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.305324 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.305325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:30.305465 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.306136 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:36:30.306137 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:30.306167 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:36:30.306190 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:30.306444 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:30.306450 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:30.306453 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:36:30.306552 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 03:36:30.306658 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":91,"threads_started":1}
I20260809 03:36:30.306983 27810 external_mini_cluster.cc:1428] Setting key 9d65c891e628bcc533888d92addbdc04
I20260809 03:36:30.307565 24998 heartbeater.cc:347] Connected to a master server at 127.27.40.190:37197
I20260809 03:36:30.307674 24998 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:30.307888 24998 heartbeater.cc:511] Master 127.27.40.190:37197 requested a full tablet report, sending...
I20260809 03:36:30.308297 24338 ts_manager.cc:194] Registered new tserver with Master: 384580df8f014e17ad05ef5016a7cd69 (127.27.40.131:45587)
I20260809 03:36:30.308692 24338 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.40.131:36345
I20260809 03:36:30.319043 27810 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:36:30.323758 24422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.320601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58638 (local address 127.27.40.190:37197)
0809 03:36:30.320673 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:30.320676 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:30.320770 (+ 94us) server_negotiation.cc:407] Connection header received
0809 03:36:30.320834 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:30.320835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:30.320846 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:30.320895 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:30.321210 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.321804 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.322600 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.322781 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.322833 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.323500 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:30.323501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:30.323501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:30.323505 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:30.323519 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:30.323523 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:30.323566 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:30.323630 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 03:36:30.323652 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:36:30.325637 24338 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:36:30.327139 24338 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:36:30.333612 25013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.329775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37730 (local address 127.27.40.129:43721)
0809 03:36:30.330004 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:30.330008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:30.330016 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:36:30.330096 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:30.330098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:30.330178 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:30.330254 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:30.330433 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.331515 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.332529 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.333134 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.333233 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.333330 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:30.333465 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 03:36:30.333503 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":138,"threads_started":1}
I20260809 03:36:30.333629 25012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.329248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:45587 (local address 127.0.0.1:45114)
0809 03:36:30.329679 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:30.329716 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:30.329813 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:30.330102 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:30.330104 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:30.330105 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:30.330168 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:30.330171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.331176 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.331177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:30.332436 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:30.332439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.333249 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.333250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:30.333338 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.333343 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:36:30.333402 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 03:36:30.333454 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":3,"thread_start_us":114,"threads_started":1}
I20260809 03:36:30.333983 25010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.329374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45114 (local address 127.27.40.131:45587)
0809 03:36:30.329566 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:30.329571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:30.329782 (+ 211us) server_negotiation.cc:407] Connection header received
0809 03:36:30.329974 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:30.329976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:30.329990 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:30.330250 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:30.330311 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.331027 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.332514 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.333142 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.333243 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.333333 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:30.333823 (+ 490us) server_negotiation.cc:299] Negotiation successful
0809 03:36:30.333878 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":94,"threads_started":1}
I20260809 03:36:30.333629 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.329513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:43721 (local address 127.0.0.1:37730)
0809 03:36:30.329776 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:30.329785 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:30.329848 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:30.330322 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:30.330322 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:30.330325 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:30.330356 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:30.330358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.331579 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.331580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:30.332417 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:30.332419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.333238 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.333239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:30.333324 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.333338 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 03:36:30.333445 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 03:36:30.333490 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":2,"thread_start_us":105,"threads_started":1}
I20260809 03:36:30.334740 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.329263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.130:35095 (local address 127.0.0.1:58228)
0809 03:36:30.329694 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:30.329725 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:30.329801 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:30.330096 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:30.330099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:30.330102 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:30.330166 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:30.330169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.331524 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.331525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:30.333466 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:30.333469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.334501 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.334502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:30.334565 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.334569 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:36:30.334633 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 03:36:30.334671 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":10,"thread_start_us":99,"threads_started":1}
I20260809 03:36:30.334776 25011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.329443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58228 (local address 127.27.40.130:35095)
0809 03:36:30.329634 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:30.329637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:30.329926 (+ 289us) server_negotiation.cc:407] Connection header received
0809 03:36:30.329971 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:30.329973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:30.329990 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:30.330072 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:30.330259 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.331432 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.333577 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.334415 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.334488 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.334565 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:30.334651 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 03:36:30.334690 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":118,"threads_started":1}
I20260809 03:36:30.334779 24520 tablet_service.cc:1467] Processing CreateTablet for tablet 3ddec341b8c84886b05c6478330fd74a (DEFAULT_TABLE table=test-table [id=1ed5e7f8b1644822beddedd2074e2134]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:30.334786 24900 tablet_service.cc:1467] Processing CreateTablet for tablet 3ddec341b8c84886b05c6478330fd74a (DEFAULT_TABLE table=test-table [id=1ed5e7f8b1644822beddedd2074e2134]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:30.335182 24900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ddec341b8c84886b05c6478330fd74a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:30.335191 24520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ddec341b8c84886b05c6478330fd74a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:30.335409 24712 tablet_service.cc:1467] Processing CreateTablet for tablet 3ddec341b8c84886b05c6478330fd74a (DEFAULT_TABLE table=test-table [id=1ed5e7f8b1644822beddedd2074e2134]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:30.335700 24712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ddec341b8c84886b05c6478330fd74a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:30.337911 25014 tablet_bootstrap.cc:492] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69: Bootstrap starting.
I20260809 03:36:30.338546 25014 tablet_bootstrap.cc:654] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:30.338853 25014 log.cc:824] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:30.339102 25015 tablet_bootstrap.cc:492] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b: Bootstrap starting.
I20260809 03:36:30.339828 25014 tablet_bootstrap.cc:492] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69: No bootstrap required, opened a new log
I20260809 03:36:30.339890 25015 tablet_bootstrap.cc:654] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:30.339919 25014 ts_tablet_manager.cc:1397] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:36:30.340221 25015 log.cc:824] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:30.340600 25016 tablet_bootstrap.cc:492] T 3ddec341b8c84886b05c6478330fd74a P 0419427f4cec4dcfbe131b65ae65467c: Bootstrap starting.
I20260809 03:36:30.341190 25015 tablet_bootstrap.cc:492] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b: No bootstrap required, opened a new log
I20260809 03:36:30.341279 25015 ts_tablet_manager.cc:1397] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:36:30.341307 25016 tablet_bootstrap.cc:654] T 3ddec341b8c84886b05c6478330fd74a P 0419427f4cec4dcfbe131b65ae65467c: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:30.341691 25016 log.cc:824] T 3ddec341b8c84886b05c6478330fd74a P 0419427f4cec4dcfbe131b65ae65467c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:30.342567 25016 tablet_bootstrap.cc:492] T 3ddec341b8c84886b05c6478330fd74a P 0419427f4cec4dcfbe131b65ae65467c: No bootstrap required, opened a new log
I20260809 03:36:30.342655 25016 ts_tablet_manager.cc:1397] T 3ddec341b8c84886b05c6478330fd74a P 0419427f4cec4dcfbe131b65ae65467c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 03:36:30.342842 25014 raft_consensus.cc:348] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6aacb354754bbea16edd30b57f3f3b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35095 } } peers { permanent_uuid: "384580df8f014e17ad05ef5016a7cd69" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45587 } } peers { permanent_uuid: "0419427f4cec4dcfbe131b65ae65467c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 43721 } }
I20260809 03:36:30.343014 25014 raft_consensus.cc:374] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:30.343050 25014 raft_consensus.cc:729] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 384580df8f014e17ad05ef5016a7cd69, State: Initialized, Role: FOLLOWER
I20260809 03:36:30.343209 25014 consensus_queue.cc:260] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6aacb354754bbea16edd30b57f3f3b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35095 } } peers { permanent_uuid: "384580df8f014e17ad05ef5016a7cd69" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45587 } } peers { permanent_uuid: "0419427f4cec4dcfbe131b65ae65467c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 43721 } }
I20260809 03:36:30.343518 25014 ts_tablet_manager.cc:1428] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 03:36:30.343551 24998 heartbeater.cc:503] Master 127.27.40.190:37197 was elected leader, sending a full tablet report...
I20260809 03:36:30.343726 25015 raft_consensus.cc:348] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6aacb354754bbea16edd30b57f3f3b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35095 } } peers { permanent_uuid: "384580df8f014e17ad05ef5016a7cd69" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45587 } } peers { permanent_uuid: "0419427f4cec4dcfbe131b65ae65467c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 43721 } }
I20260809 03:36:30.343874 25015 raft_consensus.cc:374] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:30.343914 25015 raft_consensus.cc:729] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c6aacb354754bbea16edd30b57f3f3b, State: Initialized, Role: FOLLOWER
I20260809 03:36:30.344048 25015 consensus_queue.cc:260] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6aacb354754bbea16edd30b57f3f3b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35095 } } peers { permanent_uuid: "384580df8f014e17ad05ef5016a7cd69" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45587 } } peers { permanent_uuid: "0419427f4cec4dcfbe131b65ae65467c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 43721 } }
I20260809 03:36:30.344304 25015 ts_tablet_manager.cc:1428] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:36:30.344424 24809 heartbeater.cc:503] Master 127.27.40.190:37197 was elected leader, sending a full tablet report...
I20260809 03:36:30.345260 25016 raft_consensus.cc:348] T 3ddec341b8c84886b05c6478330fd74a P 0419427f4cec4dcfbe131b65ae65467c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6aacb354754bbea16edd30b57f3f3b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35095 } } peers { permanent_uuid: "384580df8f014e17ad05ef5016a7cd69" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45587 } } peers { permanent_uuid: "0419427f4cec4dcfbe131b65ae65467c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 43721 } }
I20260809 03:36:30.345444 25016 raft_consensus.cc:374] T 3ddec341b8c84886b05c6478330fd74a P 0419427f4cec4dcfbe131b65ae65467c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:30.345489 25016 raft_consensus.cc:729] T 3ddec341b8c84886b05c6478330fd74a P 0419427f4cec4dcfbe131b65ae65467c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0419427f4cec4dcfbe131b65ae65467c, State: Initialized, Role: FOLLOWER
I20260809 03:36:30.345624 25016 consensus_queue.cc:260] T 3ddec341b8c84886b05c6478330fd74a P 0419427f4cec4dcfbe131b65ae65467c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6aacb354754bbea16edd30b57f3f3b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35095 } } peers { permanent_uuid: "384580df8f014e17ad05ef5016a7cd69" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45587 } } peers { permanent_uuid: "0419427f4cec4dcfbe131b65ae65467c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 43721 } }
I20260809 03:36:30.345842 25016 ts_tablet_manager.cc:1428] T 3ddec341b8c84886b05c6478330fd74a P 0419427f4cec4dcfbe131b65ae65467c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 03:36:30.345912 24610 heartbeater.cc:503] Master 127.27.40.190:37197 was elected leader, sending a full tablet report...
I20260809 03:36:30.386560 25022 raft_consensus.cc:482] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:36:30.386689 25022 raft_consensus.cc:504] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6aacb354754bbea16edd30b57f3f3b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35095 } } peers { permanent_uuid: "384580df8f014e17ad05ef5016a7cd69" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45587 } } peers { permanent_uuid: "0419427f4cec4dcfbe131b65ae65467c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 43721 } }
I20260809 03:36:30.387027 25022 leader_election.cc:288] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 384580df8f014e17ad05ef5016a7cd69 (127.27.40.131:45587), 0419427f4cec4dcfbe131b65ae65467c (127.27.40.129:43721)
I20260809 03:36:30.391129 25010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.387321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:47887 (local address 127.27.40.131:45587)
0809 03:36:30.387370 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:30.387372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:30.387383 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:36:30.387425 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:30.387425 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:30.387434 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:30.387527 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:30.387748 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.388534 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.390142 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.390708 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.390781 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.390789 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:30.391017 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 03:36:30.391059 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 03:36:30.391144 25013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.387325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.40.130:45473 (local address 127.27.40.129:43721)
0809 03:36:30.387370 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:30.387372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:30.387487 (+ 115us) server_negotiation.cc:407] Connection header received
0809 03:36:30.387517 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:30.387518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:30.387530 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:30.387589 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:30.387745 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.388546 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.390218 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.390725 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.390769 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.390779 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:30.391014 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 03:36:30.391065 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 03:36:30.391163 25025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.387229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:43721 (local address 127.27.40.130:45473)
0809 03:36:30.387406 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:30.387415 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:30.387470 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:30.387607 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:30.387607 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:30.387608 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:30.387662 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:30.387664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.388653 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.388654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:30.390006 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:30.390010 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.390852 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.390852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:30.390920 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.390922 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 03:36:30.390992 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 03:36:30.391036 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":64,"thread_start_us":101,"threads_started":1}
I20260809 03:36:30.391424 24811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.387229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.131:45587 (local address 127.27.40.130:47887)
0809 03:36:30.387290 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:30.387303 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:30.387351 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:30.387543 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:30.387544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:30.387546 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:30.387662 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:30.387664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.388653 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.388654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:30.390006 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:30.390010 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.390822 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.390824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:30.390902 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.390907 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:36:30.391008 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 03:36:30.391090 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":44}
I20260809 03:36:30.391489 24549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ddec341b8c84886b05c6478330fd74a" candidate_uuid: "7c6aacb354754bbea16edd30b57f3f3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0419427f4cec4dcfbe131b65ae65467c" is_pre_election: true
I20260809 03:36:30.391669 24549 raft_consensus.cc:2454] T 3ddec341b8c84886b05c6478330fd74a P 0419427f4cec4dcfbe131b65ae65467c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c6aacb354754bbea16edd30b57f3f3b in term 0.
I20260809 03:36:30.391809 24932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ddec341b8c84886b05c6478330fd74a" candidate_uuid: "7c6aacb354754bbea16edd30b57f3f3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "384580df8f014e17ad05ef5016a7cd69" is_pre_election: true
I20260809 03:36:30.391945 24644 leader_election.cc:302] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0419427f4cec4dcfbe131b65ae65467c, 7c6aacb354754bbea16edd30b57f3f3b; no voters:
I20260809 03:36:30.391975 24932 raft_consensus.cc:2454] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c6aacb354754bbea16edd30b57f3f3b in term 0.
I20260809 03:36:30.392078 25022 raft_consensus.cc:2781] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:36:30.392138 25022 raft_consensus.cc:482] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:36:30.392163 25022 raft_consensus.cc:3037] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:30.392872 25022 raft_consensus.cc:504] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6aacb354754bbea16edd30b57f3f3b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35095 } } peers { permanent_uuid: "384580df8f014e17ad05ef5016a7cd69" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45587 } } peers { permanent_uuid: "0419427f4cec4dcfbe131b65ae65467c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 43721 } }
I20260809 03:36:30.393013 25022 leader_election.cc:288] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [CANDIDATE]: Term 1 election: Requested vote from peers 384580df8f014e17ad05ef5016a7cd69 (127.27.40.131:45587), 0419427f4cec4dcfbe131b65ae65467c (127.27.40.129:43721)
I20260809 03:36:30.393193 24932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ddec341b8c84886b05c6478330fd74a" candidate_uuid: "7c6aacb354754bbea16edd30b57f3f3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "384580df8f014e17ad05ef5016a7cd69"
I20260809 03:36:30.393225 24549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ddec341b8c84886b05c6478330fd74a" candidate_uuid: "7c6aacb354754bbea16edd30b57f3f3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0419427f4cec4dcfbe131b65ae65467c"
I20260809 03:36:30.393275 24932 raft_consensus.cc:3037] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:30.393326 24549 raft_consensus.cc:3037] T 3ddec341b8c84886b05c6478330fd74a P 0419427f4cec4dcfbe131b65ae65467c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:30.394130 24549 raft_consensus.cc:2454] T 3ddec341b8c84886b05c6478330fd74a P 0419427f4cec4dcfbe131b65ae65467c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c6aacb354754bbea16edd30b57f3f3b in term 1.
I20260809 03:36:30.394173 24932 raft_consensus.cc:2454] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c6aacb354754bbea16edd30b57f3f3b in term 1.
I20260809 03:36:30.394361 24644 leader_election.cc:302] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0419427f4cec4dcfbe131b65ae65467c, 7c6aacb354754bbea16edd30b57f3f3b; no voters:
I20260809 03:36:30.394503 25022 raft_consensus.cc:2781] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:30.394642 25022 raft_consensus.cc:686] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [term 1 LEADER]: Becoming Leader. State: Replica: 7c6aacb354754bbea16edd30b57f3f3b, State: Running, Role: LEADER
I20260809 03:36:30.394742 25022 consensus_queue.cc:237] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6aacb354754bbea16edd30b57f3f3b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35095 } } peers { permanent_uuid: "384580df8f014e17ad05ef5016a7cd69" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45587 } } peers { permanent_uuid: "0419427f4cec4dcfbe131b65ae65467c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 43721 } }
I20260809 03:36:30.395642 24337 catalog_manager.cc:5495] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c6aacb354754bbea16edd30b57f3f3b (127.27.40.130). New cstate: current_term: 1 leader_uuid: "7c6aacb354754bbea16edd30b57f3f3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6aacb354754bbea16edd30b57f3f3b" member_type: VOTER last_known_addr { host: "127.27.40.130" port: 35095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "384580df8f014e17ad05ef5016a7cd69" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0419427f4cec4dcfbe131b65ae65467c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 43721 } health_report { overall_health: UNKNOWN } } }
I20260809 03:36:30.410452 25011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.407838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58238 (local address 127.27.40.130:35095)
0809 03:36:30.407912 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:30.407914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:30.407924 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:36:30.407986 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:30.407987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:30.408015 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:30.408125 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:36:30.408267 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.408740 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.409503 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:30.409752 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:30.409834 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:30.410008 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:36:30.410141 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:36:30.410232 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:30.410274 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 03:36:30.410294 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 03:36:30.412379 24932 raft_consensus.cc:1261] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69 [term 1 FOLLOWER]: Refusing update from remote peer 7c6aacb354754bbea16edd30b57f3f3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:36:30.412379 24549 raft_consensus.cc:1261] T 3ddec341b8c84886b05c6478330fd74a P 0419427f4cec4dcfbe131b65ae65467c [term 1 FOLLOWER]: Refusing update from remote peer 7c6aacb354754bbea16edd30b57f3f3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:36:30.412858 25027 consensus_queue.cc:1035] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "384580df8f014e17ad05ef5016a7cd69" member_type: VOTER last_known_addr { host: "127.27.40.131" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:36:30.413004 25022 consensus_queue.cc:1035] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0419427f4cec4dcfbe131b65ae65467c" member_type: VOTER last_known_addr { host: "127.27.40.129" port: 43721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 03:36:30.413257 24810 tablet.cc:2369] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:36:30.415755 25037 mvcc.cc:205] Tried to move back new op lower bound from 7316466034325925888 to 7316466034257534976. Current Snapshot: MvccSnapshot[applied={T|T < 7316466034325925888}]
I20260809 03:36:30.418051 24336 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:36:30.418135 24336 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:36:30.418888 24336 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ecac47575804012a1dbcb866acb0358: Sending DeleteTablet for 3 replicas of tablet 3ddec341b8c84886b05c6478330fd74a
I20260809 03:36:30.419198 24712 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ddec341b8c84886b05c6478330fd74a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:30 UTC) from {username='slave'} at 127.0.0.1:58228
I20260809 03:36:30.419308 24900 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ddec341b8c84886b05c6478330fd74a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:30 UTC) from {username='slave'} at 127.0.0.1:45114
I20260809 03:36:30.419454 25048 tablet_replica.cc:330] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b: stopping tablet replica
I20260809 03:36:30.419574 24520 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ddec341b8c84886b05c6478330fd74a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:30 UTC) from {username='slave'} at 127.0.0.1:37730
I20260809 03:36:30.419643 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 24425
I20260809 03:36:30.419559 25048 raft_consensus.cc:2229] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:36:30.427997 25048 raft_consensus.cc:2258] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:30.429804 25048 ts_tablet_manager.cc:1905] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:36:30.432613 25048 ts_tablet_manager.cc:1918] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:36:30.432803 25048 log.cc:1196] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-1/wal/wals/3ddec341b8c84886b05c6478330fd74a
I20260809 03:36:30.433332 25048 ts_tablet_manager.cc:1939] T 3ddec341b8c84886b05c6478330fd74a P 7c6aacb354754bbea16edd30b57f3f3b: Deleting consensus metadata
I20260809 03:36:30.434088 24323 catalog_manager.cc:4841] TS 7c6aacb354754bbea16edd30b57f3f3b (127.27.40.130:35095): tablet 3ddec341b8c84886b05c6478330fd74a (table test-table [id=1ed5e7f8b1644822beddedd2074e2134]) successfully deleted
I20260809 03:36:30.438758 25049 tablet_replica.cc:330] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69: stopping tablet replica
I20260809 03:36:30.438941 25049 raft_consensus.cc:2229] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:36:30.439091 25049 raft_consensus.cc:2258] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:30.439653 25049 ts_tablet_manager.cc:1905] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:36:30.441394 25049 ts_tablet_manager.cc:1918] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:36:30.441493 25049 log.cc:1196] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69: Deleting WAL directory at /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246339702443-27810-0/minicluster-data/ts-2/wal/wals/3ddec341b8c84886b05c6478330fd74a
I20260809 03:36:30.441764 25049 ts_tablet_manager.cc:1939] T 3ddec341b8c84886b05c6478330fd74a P 384580df8f014e17ad05ef5016a7cd69: Deleting consensus metadata
I20260809 03:36:30.442334 24318 catalog_manager.cc:4841] TS 384580df8f014e17ad05ef5016a7cd69 (127.27.40.131:45587): tablet 3ddec341b8c84886b05c6478330fd74a (table test-table [id=1ed5e7f8b1644822beddedd2074e2134]) successfully deleted
W20260809 03:36:30.443413 24323 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.40.129:43721 (error 108)
I20260809 03:36:30.443993 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:30.443827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.129:43721 (local address 127.0.0.1:37734)
0809 03:36:30.443885 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:30.443929 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.129:43721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 03:36:30.444059 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 24616
W20260809 03:36:30.448338 24323 catalog_manager.cc:4584] TS 0419427f4cec4dcfbe131b65ae65467c (127.27.40.129:43721): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3ddec341b8c84886b05c6478330fd74a: Network error: Client connection negotiation failed: client connection to 127.27.40.129:43721: connect: Connection refused (error 111)
I20260809 03:36:30.452200 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 24812
I20260809 03:36:30.467495 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 24290
2026-08-09T03:36:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (761 ms)
[----------] 6 tests from EncryptionPolicyTest (9244 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:30 dist-test-slave-gkw7 krb5kdc[25056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:30 dist-test-slave-gkw7 krb5kdc[25056](info): set up 2 sockets
Aug 09 03:36:30 dist-test-slave-gkw7 krb5kdc[25056](info): commencing operation
krb5kdc: starting...
W20260809 03:36:31.701138 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:31 dist-test-slave-gkw7 krb5kdc[25056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:36:31.834746 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35355
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:31.918210 25073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:31.918421 25073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:31.918442 25073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:31.919519 25073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:31.919552 25073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:31.919566 25073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:31.919579 25073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:31.919591 25073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:31.921064 25073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35355
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:31.921296 25073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:31.921507 25073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:31.926504 25086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:31.926509 25084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:31.926512 25083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:31.926698 25073 server_base.cc:1034] running on GCE node
I20260809 03:36:31.926988 25073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:31.927244 25073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:31.928418 25073 hybrid_clock.cc:648] HybridClock initialized: now 1786246591928390 us; error 43 us; skew 500 ppm
Aug 09 03:36:31 dist-test-slave-gkw7 krb5kdc[25056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246591, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:31.931069 25073 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:36:31.931515 25073 webserver.cc:460] Webserver started at http://127.0.0.1:34829/ using document root <none> and password file <none>
I20260809 03:36:31.931751 25073 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:31.931797 25073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:31.931957 25073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:31.933064 25073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "5d693f13ab7d46358e2ca754322d2c69"
format_stamp: "Formatted at 2026-08-09 03:36:31 on dist-test-slave-gkw7"
server_key: "6bb7e6cb1300c60cd3acc12bddc527a7"
server_key_iv: "df9252556458e5cbbbcb793977146230"
server_key_version: "encryptionkey@0"
I20260809 03:36:31.933437 25073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "5d693f13ab7d46358e2ca754322d2c69"
format_stamp: "Formatted at 2026-08-09 03:36:31 on dist-test-slave-gkw7"
server_key: "6bb7e6cb1300c60cd3acc12bddc527a7"
server_key_iv: "df9252556458e5cbbbcb793977146230"
server_key_version: "encryptionkey@0"
I20260809 03:36:31.934834 25073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:36:31.935581 25101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:31.935753 25073 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:36:31.935844 25073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "5d693f13ab7d46358e2ca754322d2c69"
format_stamp: "Formatted at 2026-08-09 03:36:31 on dist-test-slave-gkw7"
server_key: "6bb7e6cb1300c60cd3acc12bddc527a7"
server_key_iv: "df9252556458e5cbbbcb793977146230"
server_key_version: "encryptionkey@0"
I20260809 03:36:31.935899 25073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:31.946732 25073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:31.948578 25073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:31.948693 25073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:31.952702 25073 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35355
I20260809 03:36:31.952824 25173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35355 every 8 connection(s)
I20260809 03:36:31.953119 25073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:31.954078 25175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:31.956363 25175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69: Bootstrap starting.
I20260809 03:36:31.956956 25175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:31.957259 25175 log.cc:824] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:31.958035 25175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69: No bootstrap required, opened a new log
I20260809 03:36:31.959636 25175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d693f13ab7d46358e2ca754322d2c69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35355 } }
I20260809 03:36:31.959779 25175 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:31.959827 25175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d693f13ab7d46358e2ca754322d2c69, State: Initialized, Role: FOLLOWER
I20260809 03:36:31.960000 25175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d693f13ab7d46358e2ca754322d2c69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35355 } }
I20260809 03:36:31.960115 25175 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:31.960153 25175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:31.960210 25175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:31.960275 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 25073
I20260809 03:36:31.960392 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:31.960616 27810 external_mini_cluster.cc:1428] Setting key 419dcce1392aec26f986eb01f7ef0d8d
I20260809 03:36:31.960937 25175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d693f13ab7d46358e2ca754322d2c69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35355 } }
I20260809 03:36:31.961063 25175 leader_election.cc:302] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d693f13ab7d46358e2ca754322d2c69; no voters:
I20260809 03:36:31.961237 25175 leader_election.cc:288] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:31.961336 25179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:31.961462 25179 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [term 1 LEADER]: Becoming Leader. State: Replica: 5d693f13ab7d46358e2ca754322d2c69, State: Running, Role: LEADER
I20260809 03:36:31.961511 25175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:31.961570 25179 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d693f13ab7d46358e2ca754322d2c69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35355 } }
I20260809 03:36:31.961907 25179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d693f13ab7d46358e2ca754322d2c69. Latest consensus state: current_term: 1 leader_uuid: "5d693f13ab7d46358e2ca754322d2c69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d693f13ab7d46358e2ca754322d2c69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35355 } } }
I20260809 03:36:31.961980 25179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:31.962213 25188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:31.962239 25186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d693f13ab7d46358e2ca754322d2c69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d693f13ab7d46358e2ca754322d2c69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35355 } } }
I20260809 03:36:31.962322 25186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:31.962798 25188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:31.965103 25188 catalog_manager.cc:1347] Generated new cluster ID: 06c2560f19b64c5b97537c7ba9aedb4e
I20260809 03:36:31.965179 25188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 03:36:31 dist-test-slave-gkw7 krb5kdc[25056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 03:36:31.972497 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:31.961104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40249 (local address 127.0.0.1:35355)
0809 03:36:31.961323 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:31.961334 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:31.961374 (+ 40us) server_negotiation.cc:407] Connection header received
0809 03:36:31.961603 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:31.961611 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:31.961686 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:31.962489 (+ 803us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:31.962734 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:31.964189 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:31.965083 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:31.965711 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:31.965818 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:31.968702 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:31.968706 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:31.968706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:31.968718 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:31.970690 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:31.971170 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:31.971173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:31.971174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:31.971300 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:31.971559 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:31.971560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:31.971560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:31.971853 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:31.971998 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:31.972166 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 03:36:31.972276 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":105,"threads_started":1}
I20260809 03:36:31.973630 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:31.982779 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:31.975221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45707 (local address 127.0.0.1:35355)
0809 03:36:31.975256 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:31.975258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:31.975267 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:36:31.975316 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:31.975317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:31.975327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:31.975390 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:31.975975 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:31.976760 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:31.977798 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:31.978158 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:31.978235 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:31.979670 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:31.979672 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:31.979672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:31.979681 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:31.981314 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:31.981772 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:31.981784 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:31.981785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:31.981854 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:31.982094 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:31.982095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:31.982095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:31.982237 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:31.982359 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:31.982480 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 03:36:31.982548 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 03:36:31.983716 25188 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:31.984172 25188 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:31.998251 25188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d693f13ab7d46358e2ca754322d2c69: Generated new TSK 0
I20260809 03:36:31.998499 25188 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:32.001612 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 25073
W20260809 03:36:32.011672 25054 connection.cc:535] client connection to 127.0.0.1:35355 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35355): Connection reset by peer (error 104)
2026-08-09T03:36:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1535 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:32 dist-test-slave-gkw7 krb5kdc[25226](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:32 dist-test-slave-gkw7 krb5kdc[25226](info): set up 2 sockets
Aug 09 03:36:32 dist-test-slave-gkw7 krb5kdc[25226](info): commencing operation
krb5kdc: starting...
W20260809 03:36:33.220763 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:33 dist-test-slave-gkw7 krb5kdc[25226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:36:33.347755 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45743
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:33.428876 25240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:33.429101 25240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:33.429140 25240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:33.430361 25240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:33.430428 25240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:33.430454 25240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:33.430485 25240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:33.430510 25240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:33.431937 25240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45743
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:33.432204 25240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:33.432456 25240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:33.437402 25246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:33.437399 25249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:33.437379 25245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:33.437590 25240 server_base.cc:1034] running on GCE node
I20260809 03:36:33.437829 25240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:33.438110 25240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:33.439292 25240 hybrid_clock.cc:648] HybridClock initialized: now 1786246593439274 us; error 30 us; skew 500 ppm
Aug 09 03:36:33 dist-test-slave-gkw7 krb5kdc[25226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246593, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:33.442018 25240 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:36:33.442543 25240 webserver.cc:460] Webserver started at http://127.0.0.1:34219/ using document root <none> and password file <none>
I20260809 03:36:33.442747 25240 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:33.442791 25240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:33.442886 25240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:33.443969 25240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "7e668e435fb54389a94d46e8986e5c5a"
format_stamp: "Formatted at 2026-08-09 03:36:33 on dist-test-slave-gkw7"
server_key: "a86f8184a5d2507fc84f16fe6452e3c9"
server_key_iv: "9bbc395115f153a2d036976a3288308a"
server_key_version: "encryptionkey@0"
I20260809 03:36:33.444311 25240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "7e668e435fb54389a94d46e8986e5c5a"
format_stamp: "Formatted at 2026-08-09 03:36:33 on dist-test-slave-gkw7"
server_key: "a86f8184a5d2507fc84f16fe6452e3c9"
server_key_iv: "9bbc395115f153a2d036976a3288308a"
server_key_version: "encryptionkey@0"
I20260809 03:36:33.445714 25240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 03:36:33.446451 25262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:33.446650 25240 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:36:33.446748 25240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "7e668e435fb54389a94d46e8986e5c5a"
format_stamp: "Formatted at 2026-08-09 03:36:33 on dist-test-slave-gkw7"
server_key: "a86f8184a5d2507fc84f16fe6452e3c9"
server_key_iv: "9bbc395115f153a2d036976a3288308a"
server_key_version: "encryptionkey@0"
I20260809 03:36:33.446830 25240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:33.464449 25240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:33.466697 25240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:33.466856 25240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:33.471318 25240 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45743
I20260809 03:36:33.471390 25342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45743 every 8 connection(s)
I20260809 03:36:33.471719 25240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:33.472738 25343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:33.473122 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 25240
I20260809 03:36:33.473255 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:33.473472 27810 external_mini_cluster.cc:1428] Setting key 8245abae8ff87a55e2653cd44e78c9e3
I20260809 03:36:33.476302 25343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a: Bootstrap starting.
I20260809 03:36:33.477099 25343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:33.477424 25343 log.cc:824] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:33.478227 25343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a: No bootstrap required, opened a new log
I20260809 03:36:33.479841 25349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:33.474042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43713 (local address 127.0.0.1:45743)
0809 03:36:33.474287 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:33.474298 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:33.474350 (+ 52us) server_negotiation.cc:407] Connection header received
0809 03:36:33.474945 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:33.474955 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:33.475019 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:33.475181 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:33.475357 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:33.476875 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:33.477839 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:33.478370 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:33.478480 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:33.479153 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:33.479157 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:33.479158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:33.479327 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:33.479357 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:33.479369 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:33.479439 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:33.479565 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 03:36:33.479616 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":93,"threads_started":1}
I20260809 03:36:33.480620 25343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e668e435fb54389a94d46e8986e5c5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45743 } }
I20260809 03:36:33.480773 25343 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:33.480839 25343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e668e435fb54389a94d46e8986e5c5a, State: Initialized, Role: FOLLOWER
I20260809 03:36:33.480914 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:33.480986 25343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e668e435fb54389a94d46e8986e5c5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45743 } }
I20260809 03:36:33.481065 25343 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:33.481093 25343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:33.481130 25343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:33.481920 25343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e668e435fb54389a94d46e8986e5c5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45743 } }
I20260809 03:36:33.482041 25343 leader_election.cc:302] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e668e435fb54389a94d46e8986e5c5a; no voters:
I20260809 03:36:33.482211 25343 leader_election.cc:288] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:33.482268 25356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:33.482440 25356 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [term 1 LEADER]: Becoming Leader. State: Replica: 7e668e435fb54389a94d46e8986e5c5a, State: Running, Role: LEADER
I20260809 03:36:33.482481 25343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:33.482560 25356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e668e435fb54389a94d46e8986e5c5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45743 } }
I20260809 03:36:33.483003 25356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e668e435fb54389a94d46e8986e5c5a. Latest consensus state: current_term: 1 leader_uuid: "7e668e435fb54389a94d46e8986e5c5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e668e435fb54389a94d46e8986e5c5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45743 } } }
I20260809 03:36:33.483016 25363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e668e435fb54389a94d46e8986e5c5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e668e435fb54389a94d46e8986e5c5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45743 } } }
I20260809 03:36:33.483101 25356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:33.483139 25363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:33.483806 25374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:33.484439 25374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:33.486291 25374 catalog_manager.cc:1347] Generated new cluster ID: 95101d7d93334f04bf806bc3ee06947c
I20260809 03:36:33.486364 25374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:33.486668 25349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:33.482663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40741 (local address 127.0.0.1:45743)
0809 03:36:33.482704 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:33.482706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:33.483285 (+ 579us) server_negotiation.cc:407] Connection header received
0809 03:36:33.483370 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:33.483372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:33.483384 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:33.483453 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:33.483647 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:33.484336 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:33.485103 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:33.485354 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:33.485430 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:33.486248 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:33.486250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:33.486250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:33.486255 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:33.486272 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:33.486276 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:33.486332 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:33.486575 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 03:36:33.486589 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 03:36:33.495631 25374 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:33.496209 25374 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:33.499918 25374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e668e435fb54389a94d46e8986e5c5a: Generated new TSK 0
I20260809 03:36:33.500103 25374 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:33.505867 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 25240
W20260809 03:36:33.523231 25224 connection.cc:535] client connection to 127.0.0.1:45743 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45743): Connection reset by peer (error 104)
2026-08-09T03:36:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1511 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:33 dist-test-slave-gkw7 krb5kdc[25390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:33 dist-test-slave-gkw7 krb5kdc[25390](info): set up 2 sockets
Aug 09 03:36:33 dist-test-slave-gkw7 krb5kdc[25390](info): commencing operation
krb5kdc: starting...
W20260809 03:36:34.722786 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:34 dist-test-slave-gkw7 krb5kdc[25390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:36:34.844254 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35735
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:34.925522 25410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:34.925735 25410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:34.925766 25410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:34.926903 25410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:34.926944 25410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:34.926959 25410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:34.926972 25410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:34.927026 25410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:34.928624 25410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35735
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:34.928876 25410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:34.929114 25410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:34.933996 25417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:34.934113 25410 server_base.cc:1034] running on GCE node
W20260809 03:36:34.933951 25420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:34.933952 25416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:34.934448 25410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:34.934742 25410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:34.935925 25410 hybrid_clock.cc:648] HybridClock initialized: now 1786246594935913 us; error 41 us; skew 500 ppm
Aug 09 03:36:34 dist-test-slave-gkw7 krb5kdc[25390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246594, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:34.938522 25410 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:36:34.938995 25410 webserver.cc:460] Webserver started at http://127.0.0.1:45875/ using document root <none> and password file <none>
I20260809 03:36:34.939246 25410 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:34.939314 25410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:34.939460 25410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:34.940515 25410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "0af13901157b4ab5b41ca9fcbcead16b"
format_stamp: "Formatted at 2026-08-09 03:36:34 on dist-test-slave-gkw7"
server_key: "ba78f1eb20a1bf2dd4855ef6f92a351f"
server_key_iv: "632b7427ca7165234c01f624d07fc267"
server_key_version: "encryptionkey@0"
I20260809 03:36:34.940892 25410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "0af13901157b4ab5b41ca9fcbcead16b"
format_stamp: "Formatted at 2026-08-09 03:36:34 on dist-test-slave-gkw7"
server_key: "ba78f1eb20a1bf2dd4855ef6f92a351f"
server_key_iv: "632b7427ca7165234c01f624d07fc267"
server_key_version: "encryptionkey@0"
I20260809 03:36:34.942334 25410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:36:34.943040 25428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:34.943221 25410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:34.943311 25410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "0af13901157b4ab5b41ca9fcbcead16b"
format_stamp: "Formatted at 2026-08-09 03:36:34 on dist-test-slave-gkw7"
server_key: "ba78f1eb20a1bf2dd4855ef6f92a351f"
server_key_iv: "632b7427ca7165234c01f624d07fc267"
server_key_version: "encryptionkey@0"
I20260809 03:36:34.943389 25410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:34.946688 25410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:34.948479 25410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:34.948630 25410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:34.952574 25410 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35735
I20260809 03:36:34.952620 25500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35735 every 8 connection(s)
I20260809 03:36:34.952965 25410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:34.954042 25501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:34.956277 25501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b: Bootstrap starting.
I20260809 03:36:34.956858 25501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:34.957149 25501 log.cc:824] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:34.957870 25501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b: No bootstrap required, opened a new log
I20260809 03:36:34.959093 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 25410
I20260809 03:36:34.959240 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:34.959437 27810 external_mini_cluster.cc:1428] Setting key 9052dbc10a8b9507feaf74dcd3001f35
I20260809 03:36:34.959545 25501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af13901157b4ab5b41ca9fcbcead16b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35735 } }
I20260809 03:36:34.959707 25501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:34.959759 25501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0af13901157b4ab5b41ca9fcbcead16b, State: Initialized, Role: FOLLOWER
I20260809 03:36:34.959879 25501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af13901157b4ab5b41ca9fcbcead16b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35735 } }
I20260809 03:36:34.959980 25501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:34.960013 25501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:34.960059 25501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:34.960588 25501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af13901157b4ab5b41ca9fcbcead16b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35735 } }
I20260809 03:36:34.960708 25501 leader_election.cc:302] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0af13901157b4ab5b41ca9fcbcead16b; no voters:
I20260809 03:36:34.960901 25501 leader_election.cc:288] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:34.960943 25507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:34.961455 25507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [term 1 LEADER]: Becoming Leader. State: Replica: 0af13901157b4ab5b41ca9fcbcead16b, State: Running, Role: LEADER
I20260809 03:36:34.961581 25501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:34.961637 25507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af13901157b4ab5b41ca9fcbcead16b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35735 } }
I20260809 03:36:34.962024 25507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0af13901157b4ab5b41ca9fcbcead16b. Latest consensus state: current_term: 1 leader_uuid: "0af13901157b4ab5b41ca9fcbcead16b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af13901157b4ab5b41ca9fcbcead16b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35735 } } }
I20260809 03:36:34.962344 25506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:34.959987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36973 (local address 127.0.0.1:35735)
0809 03:36:34.960178 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:34.960187 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:34.960221 (+ 34us) server_negotiation.cc:407] Connection header received
0809 03:36:34.960541 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:34.960548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:34.960572 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:34.960719 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:34.961442 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:34.961445 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:34.961446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:34.961450 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:34.961494 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:34.961509 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:34.961640 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:34.961898 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 03:36:34.961942 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
I20260809 03:36:34.962411 25507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:34.962034 25510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0af13901157b4ab5b41ca9fcbcead16b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af13901157b4ab5b41ca9fcbcead16b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35735 } } }
I20260809 03:36:34.963363 25510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b [sys.catalog]: This master's current role is: LEADER
W20260809 03:36:34.963586 25533 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:36:34.963650 25533 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:36:34.963717 25525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:34.963835 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:34.963869 25525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:34.965413 25525 catalog_manager.cc:1347] Generated new cluster ID: dd74dd9f938d4e25888b48a6c6d36e17
I20260809 03:36:34.965466 25525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:34.966516 25506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:34.965397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35321 (local address 127.0.0.1:35735)
0809 03:36:34.965429 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:34.965431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:34.965554 (+ 123us) server_negotiation.cc:407] Connection header received
0809 03:36:34.965664 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:34.965665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:34.965671 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:34.965734 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:34.966305 (+ 571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:34.966307 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:34.966307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:34.966310 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:34.966323 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:34.966327 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:34.966377 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:34.966449 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 03:36:34.966461 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 03:36:34.989027 25525 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:34.989250 25525 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:34.999728 25525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0af13901157b4ab5b41ca9fcbcead16b: Generated new TSK 0
I20260809 03:36:34.999946 25525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:35.008558 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 25410
2026-08-09T03:36:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1499 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:35 dist-test-slave-gkw7 krb5kdc[25547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:35 dist-test-slave-gkw7 krb5kdc[25547](info): set up 2 sockets
Aug 09 03:36:35 dist-test-slave-gkw7 krb5kdc[25547](info): commencing operation
krb5kdc: starting...
W20260809 03:36:36.302574 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:36 dist-test-slave-gkw7 krb5kdc[25547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:36:36.427994 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36905
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:36.510797 25565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:36.510998 25565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:36.511020 25565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:36.512099 25565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:36.512146 25565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:36.512161 25565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:36.512173 25565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:36.512185 25565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:36.513613 25565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:36.513810 25565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:36.513976 25565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:36.518893 25573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:36.518869 25571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:36.518896 25575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:36.518909 25565 server_base.cc:1034] running on GCE node
I20260809 03:36:36.519384 25565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:36.519706 25565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:36.520882 25565 hybrid_clock.cc:648] HybridClock initialized: now 1786246596520874 us; error 35 us; skew 500 ppm
Aug 09 03:36:36 dist-test-slave-gkw7 krb5kdc[25547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246596, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:36.523352 25565 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:36:36.523823 25565 webserver.cc:460] Webserver started at http://127.0.0.1:35171/ using document root <none> and password file <none>
I20260809 03:36:36.524009 25565 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:36.524047 25565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:36.524137 25565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:36.525156 25565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "d57e8bab2eb04befa9e746c310a8efa3"
format_stamp: "Formatted at 2026-08-09 03:36:36 on dist-test-slave-gkw7"
server_key: "ce3a18b0f1bb9f1f5dcdba104658a583"
server_key_iv: "f6f177592bfefb2c2c462487649ae271"
server_key_version: "encryptionkey@0"
I20260809 03:36:36.525465 25565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "d57e8bab2eb04befa9e746c310a8efa3"
format_stamp: "Formatted at 2026-08-09 03:36:36 on dist-test-slave-gkw7"
server_key: "ce3a18b0f1bb9f1f5dcdba104658a583"
server_key_iv: "f6f177592bfefb2c2c462487649ae271"
server_key_version: "encryptionkey@0"
I20260809 03:36:36.526934 25565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:36:36.527645 25586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:36.527863 25565 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:36:36.527961 25565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "d57e8bab2eb04befa9e746c310a8efa3"
format_stamp: "Formatted at 2026-08-09 03:36:36 on dist-test-slave-gkw7"
server_key: "ce3a18b0f1bb9f1f5dcdba104658a583"
server_key_iv: "f6f177592bfefb2c2c462487649ae271"
server_key_version: "encryptionkey@0"
I20260809 03:36:36.528023 25565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:36.546584 25565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:36.548540 25565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:36.548650 25565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:36.552588 25565 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36905
I20260809 03:36:36.552649 25662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36905 every 8 connection(s)
I20260809 03:36:36.553090 25565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:36.554067 25663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:36.556392 25663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3: Bootstrap starting.
I20260809 03:36:36.557011 25663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:36.557322 25663 log.cc:824] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:36.558178 25663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3: No bootstrap required, opened a new log
I20260809 03:36:36.559788 25663 raft_consensus.cc:348] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d57e8bab2eb04befa9e746c310a8efa3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } }
I20260809 03:36:36.559949 25663 raft_consensus.cc:374] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:36.560000 25663 raft_consensus.cc:729] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d57e8bab2eb04befa9e746c310a8efa3, State: Initialized, Role: FOLLOWER
I20260809 03:36:36.560178 25663 consensus_queue.cc:260] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d57e8bab2eb04befa9e746c310a8efa3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } }
I20260809 03:36:36.560262 25663 raft_consensus.cc:388] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:36.560302 25663 raft_consensus.cc:482] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:36.560354 25663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:36.560915 25663 raft_consensus.cc:504] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d57e8bab2eb04befa9e746c310a8efa3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } }
I20260809 03:36:36.561038 25663 leader_election.cc:302] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d57e8bab2eb04befa9e746c310a8efa3; no voters:
I20260809 03:36:36.561231 25663 leader_election.cc:288] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:36.561285 25668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:36.561498 25663 sys_catalog.cc:564] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:36.561506 25668 raft_consensus.cc:686] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [term 1 LEADER]: Becoming Leader. State: Replica: d57e8bab2eb04befa9e746c310a8efa3, State: Running, Role: LEADER
I20260809 03:36:36.561724 25668 consensus_queue.cc:237] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d57e8bab2eb04befa9e746c310a8efa3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } }
I20260809 03:36:36.562137 25671 sys_catalog.cc:455] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d57e8bab2eb04befa9e746c310a8efa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d57e8bab2eb04befa9e746c310a8efa3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } } }
I20260809 03:36:36.562244 25671 sys_catalog.cc:458] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:36.562140 25673 sys_catalog.cc:455] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d57e8bab2eb04befa9e746c310a8efa3. Latest consensus state: current_term: 1 leader_uuid: "d57e8bab2eb04befa9e746c310a8efa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d57e8bab2eb04befa9e746c310a8efa3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } } }
I20260809 03:36:36.562381 25673 sys_catalog.cc:458] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:36.562541 25675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:36.563263 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 25565
I20260809 03:36:36.563369 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:36.563542 27810 external_mini_cluster.cc:1428] Setting key e410329adb91b53577e7903a6c728fa9
I20260809 03:36:36.563850 25675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:36.565450 25675 catalog_manager.cc:1347] Generated new cluster ID: 80d757e7a95b4980b408ff16a23ef6c8
I20260809 03:36:36.565511 25675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:36.575220 25675 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:36.575941 25675 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 03:36:36 dist-test-slave-gkw7 krb5kdc[25547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 03:36:36.580400 25694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:36.564308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53333 (local address 127.0.0.1:36905)
0809 03:36:36.569868 (+ 5560us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:36.569877 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:36.569907 (+ 30us) server_negotiation.cc:407] Connection header received
0809 03:36:36.570018 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:36.570025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:36.570088 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:36.570205 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:36.571065 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:36.572473 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:36.573675 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:36.574281 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:36.574389 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:36.576994 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:36.576998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:36.576998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:36.577010 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:36.578755 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:36.579153 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:36.579156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:36.579156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:36.579247 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:36.579461 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:36.579462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:36.579462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:36.579682 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:36.579841 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:36.579956 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 03:36:36.580220 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5508,"thread_start_us":84,"threads_started":1}
I20260809 03:36:36.581317 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:36.582144 25675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d57e8bab2eb04befa9e746c310a8efa3: Generated new TSK 0
I20260809 03:36:36.582302 25675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:36.589195 25694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:36.582960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45765 (local address 127.0.0.1:36905)
0809 03:36:36.582993 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:36.582995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:36.583119 (+ 124us) server_negotiation.cc:407] Connection header received
0809 03:36:36.583192 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:36.583193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:36.583206 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:36.583254 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:36.583767 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:36.584260 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:36.585251 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:36.585488 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:36.585593 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:36.586741 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:36.586742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:36.586742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:36.586749 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:36.588005 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:36.588370 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:36.588371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:36.588372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:36.588424 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:36.588637 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:36.588637 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:36.588638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:36.588747 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:36.588870 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:36.588977 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 03:36:36.589098 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 03:36:36.591048 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 25565
2026-08-09T03:36:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1580 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:36 dist-test-slave-gkw7 krb5kdc[25709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:36 dist-test-slave-gkw7 krb5kdc[25709](info): set up 2 sockets
Aug 09 03:36:36 dist-test-slave-gkw7 krb5kdc[25709](info): commencing operation
krb5kdc: starting...
W20260809 03:36:37.928287 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:38 dist-test-slave-gkw7 krb5kdc[25709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:36:38.055719 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35203
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:38.138932 25733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:38.139194 25733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:38.139235 25733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:38.140378 25733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:38.140456 25733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:38.140480 25733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:38.140503 25733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:38.140533 25733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:38.142064 25733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:38.142347 25733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:38.142586 25733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:38.147614 25740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:38.147640 25742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:38.147624 25739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:38.147856 25733 server_base.cc:1034] running on GCE node
I20260809 03:36:38.148093 25733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:38.148355 25733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:38.149501 25733 hybrid_clock.cc:648] HybridClock initialized: now 1786246598149489 us; error 28 us; skew 500 ppm
Aug 09 03:36:38 dist-test-slave-gkw7 krb5kdc[25709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246598, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:38.157603 25733 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:36:38.158051 25733 webserver.cc:460] Webserver started at http://127.0.0.1:46345/ using document root <none> and password file <none>
I20260809 03:36:38.158326 25733 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:38.158375 25733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:38.158547 25733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:38.159639 25733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "8eb79f21910a4138beefe91e5585a8ed"
format_stamp: "Formatted at 2026-08-09 03:36:38 on dist-test-slave-gkw7"
server_key: "d7d0d3a4880d5c2e7f764e105519093a"
server_key_iv: "624ef3da8c7c86b9da6c777113d92f74"
server_key_version: "encryptionkey@0"
I20260809 03:36:38.160004 25733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "8eb79f21910a4138beefe91e5585a8ed"
format_stamp: "Formatted at 2026-08-09 03:36:38 on dist-test-slave-gkw7"
server_key: "d7d0d3a4880d5c2e7f764e105519093a"
server_key_iv: "624ef3da8c7c86b9da6c777113d92f74"
server_key_version: "encryptionkey@0"
I20260809 03:36:38.161429 25733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 03:36:38.162235 25750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:38.162438 25733 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:38.162536 25733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "8eb79f21910a4138beefe91e5585a8ed"
format_stamp: "Formatted at 2026-08-09 03:36:38 on dist-test-slave-gkw7"
server_key: "d7d0d3a4880d5c2e7f764e105519093a"
server_key_iv: "624ef3da8c7c86b9da6c777113d92f74"
server_key_version: "encryptionkey@0"
I20260809 03:36:38.162611 25733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:38.176568 25733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:38.178637 25733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:38.178815 25733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:38.183235 25733 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35203
I20260809 03:36:38.183295 25816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35203 every 8 connection(s)
I20260809 03:36:38.183638 25733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:38.184798 25819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:38.187328 25819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed: Bootstrap starting.
I20260809 03:36:38.187950 25819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:38.188252 25819 log.cc:824] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:38.188998 25819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed: No bootstrap required, opened a new log
I20260809 03:36:38.190678 25819 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb79f21910a4138beefe91e5585a8ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35203 } }
I20260809 03:36:38.190817 25819 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:38.190841 25819 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb79f21910a4138beefe91e5585a8ed, State: Initialized, Role: FOLLOWER
I20260809 03:36:38.191030 25819 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb79f21910a4138beefe91e5585a8ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35203 } }
I20260809 03:36:38.191113 25819 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:38.191157 25819 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:38.191207 25819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:38.191412 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 25733
I20260809 03:36:38.191550 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:38.191745 27810 external_mini_cluster.cc:1428] Setting key fdfaf98ea2277604555c643a7f332310
I20260809 03:36:38.191870 25819 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb79f21910a4138beefe91e5585a8ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35203 } }
I20260809 03:36:38.192000 25819 leader_election.cc:302] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8eb79f21910a4138beefe91e5585a8ed; no voters:
I20260809 03:36:38.192170 25819 leader_election.cc:288] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:38.192219 25828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:38.192683 25828 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [term 1 LEADER]: Becoming Leader. State: Replica: 8eb79f21910a4138beefe91e5585a8ed, State: Running, Role: LEADER
I20260809 03:36:38.192773 25819 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:38.192828 25828 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb79f21910a4138beefe91e5585a8ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35203 } }
I20260809 03:36:38.193289 25828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8eb79f21910a4138beefe91e5585a8ed. Latest consensus state: current_term: 1 leader_uuid: "8eb79f21910a4138beefe91e5585a8ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb79f21910a4138beefe91e5585a8ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35203 } } }
I20260809 03:36:38.193413 25828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:38.193392 25832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8eb79f21910a4138beefe91e5585a8ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb79f21910a4138beefe91e5585a8ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35203 } } }
I20260809 03:36:38.193554 25832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed [sys.catalog]: This master's current role is: LEADER
W20260809 03:36:38.195168 25852 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:36:38.195226 25852 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:36:38.195283 25837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:38.195425 25837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:38.197381 25837 catalog_manager.cc:1347] Generated new cluster ID: 1a4ff57e1ec642a4aff3c7ba345a4c98
I20260809 03:36:38.197435 25837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:38.198240 25831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:38.192263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48895 (local address 127.0.0.1:35203)
0809 03:36:38.192471 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:38.192482 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:38.192522 (+ 40us) server_negotiation.cc:407] Connection header received
0809 03:36:38.192664 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:38.192672 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:38.192736 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:38.193140 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:38.193257 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:38.194794 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:38.195937 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:38.196533 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:38.196661 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:38.197367 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:38.197371 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:38.197371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:38.197634 (+ 263us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:38.197671 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:38.197686 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:38.197755 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:38.197853 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 03:36:38.198020 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":106,"threads_started":1}
I20260809 03:36:38.199165 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:38.204176 25831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:38.200711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36357 (local address 127.0.0.1:35203)
0809 03:36:38.200757 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:38.200758 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:38.200919 (+ 161us) server_negotiation.cc:407] Connection header received
0809 03:36:38.200948 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:38.200949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:38.200960 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:38.201025 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:38.201150 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:38.201928 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:38.202885 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:38.203187 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:38.203266 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:38.203934 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:38.203935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:38.203935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:38.203941 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:38.203957 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:38.203961 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:38.204023 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:38.204077 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 03:36:38.204103 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 03:36:38.216192 25837 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:38.216742 25837 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:38.233579 25837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8eb79f21910a4138beefe91e5585a8ed: Generated new TSK 0
I20260809 03:36:38.233822 25837 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:38.239959 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 25733
2026-08-09T03:36:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1650 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:38 dist-test-slave-gkw7 krb5kdc[25870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:38 dist-test-slave-gkw7 krb5kdc[25870](info): set up 2 sockets
Aug 09 03:36:38 dist-test-slave-gkw7 krb5kdc[25870](info): commencing operation
krb5kdc: starting...
W20260809 03:36:39.522027 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:39 dist-test-slave-gkw7 krb5kdc[25870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:36:39.644454 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33273
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:39.726588 25887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:39.726785 25887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:39.726809 25887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:39.727969 25887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:39.728010 25887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:39.728025 25887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:39.728037 25887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:39.728089 25887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:39.729635 25887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:39.729882 25887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:39.730106 25887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:39.735138 25897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:39.735136 25895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:39.735133 25894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:39.735412 25887 server_base.cc:1034] running on GCE node
I20260809 03:36:39.735703 25887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:39.736006 25887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:39.737190 25887 hybrid_clock.cc:648] HybridClock initialized: now 1786246599737174 us; error 44 us; skew 500 ppm
Aug 09 03:36:39 dist-test-slave-gkw7 krb5kdc[25870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246599, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:39.739789 25887 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:36:39.740262 25887 webserver.cc:460] Webserver started at http://127.0.0.1:34151/ using document root <none> and password file <none>
I20260809 03:36:39.740489 25887 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:39.740535 25887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:39.740693 25887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:39.741884 25887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "790ac88ff31142aa8e2dd1b0c59bac92"
format_stamp: "Formatted at 2026-08-09 03:36:39 on dist-test-slave-gkw7"
server_key: "d2500ef69a4ff3aa4367d46dccb71cbe"
server_key_iv: "8383370cf25db5e1c3f18223425cd2db"
server_key_version: "encryptionkey@0"
I20260809 03:36:39.742247 25887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "790ac88ff31142aa8e2dd1b0c59bac92"
format_stamp: "Formatted at 2026-08-09 03:36:39 on dist-test-slave-gkw7"
server_key: "d2500ef69a4ff3aa4367d46dccb71cbe"
server_key_iv: "8383370cf25db5e1c3f18223425cd2db"
server_key_version: "encryptionkey@0"
I20260809 03:36:39.743592 25887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:39.744314 25905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:39.744493 25887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:36:39.744591 25887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "790ac88ff31142aa8e2dd1b0c59bac92"
format_stamp: "Formatted at 2026-08-09 03:36:39 on dist-test-slave-gkw7"
server_key: "d2500ef69a4ff3aa4367d46dccb71cbe"
server_key_iv: "8383370cf25db5e1c3f18223425cd2db"
server_key_version: "encryptionkey@0"
I20260809 03:36:39.744668 25887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:39.748092 25887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:39.750280 25887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:39.750432 25887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:39.754544 25887 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33273
I20260809 03:36:39.754585 25978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33273 every 8 connection(s)
I20260809 03:36:39.754932 25887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:39.755918 25981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:39.758350 25981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92: Bootstrap starting.
I20260809 03:36:39.758952 25981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:39.759274 25981 log.cc:824] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:39.759354 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 25887
I20260809 03:36:39.759506 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:39.759729 27810 external_mini_cluster.cc:1428] Setting key f87a24dcb065d980694dfe47e69d3694
I20260809 03:36:39.760140 25981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92: No bootstrap required, opened a new log
I20260809 03:36:39.762537 25981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790ac88ff31142aa8e2dd1b0c59bac92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33273 } }
I20260809 03:36:39.762727 25981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:39.762776 25981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 790ac88ff31142aa8e2dd1b0c59bac92, State: Initialized, Role: FOLLOWER
I20260809 03:36:39.762902 25981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790ac88ff31142aa8e2dd1b0c59bac92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33273 } }
I20260809 03:36:39.762975 25985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:39.760257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47835 (local address 127.0.0.1:33273)
0809 03:36:39.760434 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:39.760448 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:39.760479 (+ 31us) server_negotiation.cc:407] Connection header received
0809 03:36:39.761397 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:39.761404 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:39.761431 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:39.761614 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:39.762332 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:39.762336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:39.762336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:39.762339 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:39.762376 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:39.762386 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:39.762440 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:39.762532 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 03:36:39.762563 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260809 03:36:39.763023 25981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:39.763079 25981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:39.763128 25981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:39.763867 25981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790ac88ff31142aa8e2dd1b0c59bac92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33273 } }
I20260809 03:36:39.763995 25981 leader_election.cc:302] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 790ac88ff31142aa8e2dd1b0c59bac92; no voters:
I20260809 03:36:39.764082 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:39.764163 25981 leader_election.cc:288] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:39.764248 25986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:39.764401 25981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:39.764431 25986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [term 1 LEADER]: Becoming Leader. State: Replica: 790ac88ff31142aa8e2dd1b0c59bac92, State: Running, Role: LEADER
I20260809 03:36:39.764628 25986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790ac88ff31142aa8e2dd1b0c59bac92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33273 } }
I20260809 03:36:39.765021 25988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "790ac88ff31142aa8e2dd1b0c59bac92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790ac88ff31142aa8e2dd1b0c59bac92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33273 } } }
I20260809 03:36:39.765130 25988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:39.765285 25989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 790ac88ff31142aa8e2dd1b0c59bac92. Latest consensus state: current_term: 1 leader_uuid: "790ac88ff31142aa8e2dd1b0c59bac92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790ac88ff31142aa8e2dd1b0c59bac92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33273 } } }
I20260809 03:36:39.765337 25989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:39.765671 26002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:39.766230 26002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:39.767735 25985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:39.766624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60901 (local address 127.0.0.1:33273)
0809 03:36:39.766662 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:39.766663 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:39.766782 (+ 119us) server_negotiation.cc:407] Connection header received
0809 03:36:39.766840 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:39.766841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:39.766846 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:39.766904 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:39.767514 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:39.767516 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:39.767516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:39.767521 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:39.767534 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:39.767538 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:39.767597 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:39.767655 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 03:36:39.767667 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 03:36:39.767855 26002 catalog_manager.cc:1347] Generated new cluster ID: 32df2d38f70d4799bf991655c86d6d01
I20260809 03:36:39.767904 26002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:39.786255 26002 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:39.786439 26002 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:39.796398 26002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 790ac88ff31142aa8e2dd1b0c59bac92: Generated new TSK 0
I20260809 03:36:39.796599 26002 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:39.798623 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 25887
2026-08-09T03:36:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1556 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:39 dist-test-slave-gkw7 krb5kdc[26024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:39 dist-test-slave-gkw7 krb5kdc[26024](info): set up 2 sockets
Aug 09 03:36:39 dist-test-slave-gkw7 krb5kdc[26024](info): commencing operation
krb5kdc: starting...
W20260809 03:36:41.032480 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:41 dist-test-slave-gkw7 krb5kdc[26024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:36:41.156215 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34005
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:41.237316 26040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:41.237593 26040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:41.237634 26040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:41.238777 26040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:41.238852 26040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:41.238876 26040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:41.238897 26040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:41.238926 26040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:41.240430 26040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:41.240716 26040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:41.240957 26040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:41.245886 26046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:41.245931 26047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:41.246021 26040 server_base.cc:1034] running on GCE node
W20260809 03:36:41.245885 26049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:41.246460 26040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:41.246767 26040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:41.247934 26040 hybrid_clock.cc:648] HybridClock initialized: now 1786246601247925 us; error 42 us; skew 500 ppm
Aug 09 03:36:41 dist-test-slave-gkw7 krb5kdc[26024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246601, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:41.250445 26040 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:36:41.250931 26040 webserver.cc:460] Webserver started at http://127.0.0.1:33025/ using document root <none> and password file <none>
I20260809 03:36:41.251171 26040 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:41.251247 26040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:41.251405 26040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:41.252480 26040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "52aca77a28304c388d04a7928958dca7"
format_stamp: "Formatted at 2026-08-09 03:36:41 on dist-test-slave-gkw7"
server_key: "e5a394ba1cab7cf1ea1211755c31bd54"
server_key_iv: "32d9f4acbc121d67f809aa37c68ea474"
server_key_version: "encryptionkey@0"
I20260809 03:36:41.252861 26040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "52aca77a28304c388d04a7928958dca7"
format_stamp: "Formatted at 2026-08-09 03:36:41 on dist-test-slave-gkw7"
server_key: "e5a394ba1cab7cf1ea1211755c31bd54"
server_key_iv: "32d9f4acbc121d67f809aa37c68ea474"
server_key_version: "encryptionkey@0"
I20260809 03:36:41.254391 26040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:36:41.255096 26059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:41.255308 26040 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:36:41.255420 26040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "52aca77a28304c388d04a7928958dca7"
format_stamp: "Formatted at 2026-08-09 03:36:41 on dist-test-slave-gkw7"
server_key: "e5a394ba1cab7cf1ea1211755c31bd54"
server_key_iv: "32d9f4acbc121d67f809aa37c68ea474"
server_key_version: "encryptionkey@0"
I20260809 03:36:41.255498 26040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:41.299937 26040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:41.301978 26040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:41.302156 26040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:41.306293 26040 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34005
I20260809 03:36:41.306331 26134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34005 every 8 connection(s)
I20260809 03:36:41.306721 26040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:41.307646 26135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:41.309954 26135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7: Bootstrap starting.
I20260809 03:36:41.310572 26135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:41.310864 26135 log.cc:824] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:41.311589 26135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7: No bootstrap required, opened a new log
I20260809 03:36:41.311595 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 26040
I20260809 03:36:41.311746 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:41.311957 27810 external_mini_cluster.cc:1428] Setting key cf89be90368156dbc0383b5f761b977e
I20260809 03:36:41.314005 26135 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52aca77a28304c388d04a7928958dca7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34005 } }
I20260809 03:36:41.314133 26135 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:41.314157 26135 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52aca77a28304c388d04a7928958dca7, State: Initialized, Role: FOLLOWER
I20260809 03:36:41.314304 26135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52aca77a28304c388d04a7928958dca7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34005 } }
I20260809 03:36:41.314370 26135 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:41.314419 26135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:41.314471 26135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:41.315246 26135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52aca77a28304c388d04a7928958dca7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34005 } }
I20260809 03:36:41.315366 26135 leader_election.cc:302] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52aca77a28304c388d04a7928958dca7; no voters:
I20260809 03:36:41.315569 26135 leader_election.cc:288] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:41.315593 26140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:41.315799 26140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [term 1 LEADER]: Becoming Leader. State: Replica: 52aca77a28304c388d04a7928958dca7, State: Running, Role: LEADER
I20260809 03:36:41.315953 26135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:41.315986 26140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52aca77a28304c388d04a7928958dca7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34005 } }
I20260809 03:36:41.316409 26140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52aca77a28304c388d04a7928958dca7. Latest consensus state: current_term: 1 leader_uuid: "52aca77a28304c388d04a7928958dca7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52aca77a28304c388d04a7928958dca7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34005 } } }
I20260809 03:36:41.316659 26140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:41.316584 26141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52aca77a28304c388d04a7928958dca7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52aca77a28304c388d04a7928958dca7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34005 } } }
I20260809 03:36:41.317090 26141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:41.317086 26151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:41.317871 26151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:41.319619 26151 catalog_manager.cc:1347] Generated new cluster ID: 2a6b5b692f994ed8b4d496e776afc802
I20260809 03:36:41.319672 26151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 03:36:41 dist-test-slave-gkw7 krb5kdc[26024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 03:36:41.324016 26139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:41.312623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:48399 (local address 127.0.0.1:34005)
0809 03:36:41.312837 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:41.312847 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:41.312882 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:36:41.313573 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:41.313583 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:41.313648 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:41.313816 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:41.314489 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:41.315506 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:41.316604 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:41.317223 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:41.317312 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:41.320392 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:41.320396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:41.320396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:41.320409 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:41.322332 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:41.322788 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:41.322791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:41.322791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:41.322891 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:41.323139 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:41.323140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:41.323141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:41.323421 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:41.323569 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:41.323736 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 03:36:41.323828 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":94,"threads_started":1}
I20260809 03:36:41.324980 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:41.331059 26151 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:41.331772 26151 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:41.333942 26139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:41.326591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:42417 (local address 127.0.0.1:34005)
0809 03:36:41.326632 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:41.326634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:41.326730 (+ 96us) server_negotiation.cc:407] Connection header received
0809 03:36:41.326773 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:41.326774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:41.326783 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:41.326836 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:41.327464 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:41.328261 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:41.329270 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:41.329625 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:41.329690 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:41.331132 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:41.331133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:41.331133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:41.331140 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:41.332706 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:41.333131 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:41.333132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:41.333133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:41.333187 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:41.333422 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:41.333422 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:41.333423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:41.333556 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:41.333668 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:41.333797 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 03:36:41.333856 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 03:36:41.336977 26151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52aca77a28304c388d04a7928958dca7: Generated new TSK 0
I20260809 03:36:41.337182 26151 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:41.345484 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 26040
W20260809 03:36:41.359965 26022 connection.cc:535] client connection to 127.0.0.1:34005 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34005): Connection reset by peer (error 104)
2026-08-09T03:36:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1549 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:41 dist-test-slave-gkw7 krb5kdc[26175](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:41 dist-test-slave-gkw7 krb5kdc[26175](info): set up 2 sockets
Aug 09 03:36:41 dist-test-slave-gkw7 krb5kdc[26175](info): commencing operation
krb5kdc: starting...
W20260809 03:36:42.589993 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:42 dist-test-slave-gkw7 krb5kdc[26175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:36:42.711469 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34293
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:42.793653 26193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:42.793895 26193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:42.793932 26193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:42.795064 26193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:42.795130 26193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:42.795157 26193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:42.795187 26193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:42.795219 26193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:42.796700 26193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:42.796972 26193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:42.797190 26193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:42.802088 26199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:42.802090 26200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:42.802091 26204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:42.802316 26193 server_base.cc:1034] running on GCE node
I20260809 03:36:42.802539 26193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:42.802850 26193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:42.804019 26193 hybrid_clock.cc:648] HybridClock initialized: now 1786246602804004 us; error 44 us; skew 500 ppm
Aug 09 03:36:42 dist-test-slave-gkw7 krb5kdc[26175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246602, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:42.806586 26193 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:36:42.807055 26193 webserver.cc:460] Webserver started at http://127.0.0.1:39115/ using document root <none> and password file <none>
I20260809 03:36:42.807250 26193 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:42.807292 26193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:42.807384 26193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:42.808395 26193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "3d015e6937b449df9bdb25ad9fc121e1"
format_stamp: "Formatted at 2026-08-09 03:36:42 on dist-test-slave-gkw7"
server_key: "53c6bc252db917f7387c52750c0e2d1f"
server_key_iv: "092d3300df831cc7bce5a86d01d1a108"
server_key_version: "encryptionkey@0"
I20260809 03:36:42.808713 26193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "3d015e6937b449df9bdb25ad9fc121e1"
format_stamp: "Formatted at 2026-08-09 03:36:42 on dist-test-slave-gkw7"
server_key: "53c6bc252db917f7387c52750c0e2d1f"
server_key_iv: "092d3300df831cc7bce5a86d01d1a108"
server_key_version: "encryptionkey@0"
I20260809 03:36:42.810082 26193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 03:36:42.810786 26212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:42.810966 26193 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:36:42.811059 26193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "3d015e6937b449df9bdb25ad9fc121e1"
format_stamp: "Formatted at 2026-08-09 03:36:42 on dist-test-slave-gkw7"
server_key: "53c6bc252db917f7387c52750c0e2d1f"
server_key_iv: "092d3300df831cc7bce5a86d01d1a108"
server_key_version: "encryptionkey@0"
I20260809 03:36:42.811128 26193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:42.819785 26193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:42.821470 26193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:42.821662 26193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:42.825788 26193 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34293
I20260809 03:36:42.825834 26287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34293 every 8 connection(s)
I20260809 03:36:42.826159 26193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:42.826982 26288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:42.829187 26288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1: Bootstrap starting.
I20260809 03:36:42.829838 26288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:42.830127 26288 log.cc:824] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:42.830812 26288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1: No bootstrap required, opened a new log
I20260809 03:36:42.832404 26288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d015e6937b449df9bdb25ad9fc121e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } }
I20260809 03:36:42.832537 26288 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:42.832581 26288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d015e6937b449df9bdb25ad9fc121e1, State: Initialized, Role: FOLLOWER
I20260809 03:36:42.832748 26288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d015e6937b449df9bdb25ad9fc121e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } }
I20260809 03:36:42.832815 26288 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:42.832851 26288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:42.832901 26288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:42.833416 26288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d015e6937b449df9bdb25ad9fc121e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } }
I20260809 03:36:42.833523 26288 leader_election.cc:302] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d015e6937b449df9bdb25ad9fc121e1; no voters:
I20260809 03:36:42.833725 26288 leader_election.cc:288] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:42.833770 26296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:42.833940 26296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3d015e6937b449df9bdb25ad9fc121e1, State: Running, Role: LEADER
I20260809 03:36:42.834028 26288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:42.834098 26296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d015e6937b449df9bdb25ad9fc121e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } }
I20260809 03:36:42.834489 26297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d015e6937b449df9bdb25ad9fc121e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d015e6937b449df9bdb25ad9fc121e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } } }
I20260809 03:36:42.834501 26298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d015e6937b449df9bdb25ad9fc121e1. Latest consensus state: current_term: 1 leader_uuid: "3d015e6937b449df9bdb25ad9fc121e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d015e6937b449df9bdb25ad9fc121e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } } }
I20260809 03:36:42.834645 26298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:42.834858 26297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:42.835037 26300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:42.835592 26300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:42.836400 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 26193
I20260809 03:36:42.836525 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:42.836746 27810 external_mini_cluster.cc:1428] Setting key 79ec960f07933ddd1256785f26240735
I20260809 03:36:42.837263 26300 catalog_manager.cc:1347] Generated new cluster ID: 91f8c5d81789465f9884290bf0d84a7e
I20260809 03:36:42.837316 26300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:42.842320 26300 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:42.843106 26300 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:42.843133 26319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:42.837213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:60395 (local address 127.0.0.1:34293)
0809 03:36:42.837631 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:42.837640 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:42.837670 (+ 30us) server_negotiation.cc:407] Connection header received
0809 03:36:42.837785 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:42.837792 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:42.837849 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:42.837989 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:42.838241 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:42.839699 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:42.840763 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:42.841420 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:42.841573 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:42.842407 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:42.842411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:42.842411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:42.842597 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:42.842627 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:42.842639 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:42.842732 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:42.842815 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 03:36:42.842974 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":371,"thread_start_us":76,"threads_started":1}
I20260809 03:36:42.844147 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:42.851148 26300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d015e6937b449df9bdb25ad9fc121e1: Generated new TSK 0
I20260809 03:36:42.851326 26300 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:42.852746 26319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:42.845864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:48349 (local address 127.0.0.1:34293)
0809 03:36:42.845901 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:42.845904 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:42.849694 (+ 3790us) server_negotiation.cc:407] Connection header received
0809 03:36:42.849779 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:42.849780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:42.849791 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:42.849860 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:42.850016 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:42.850559 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:42.851589 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:42.851796 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:42.851867 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:42.852517 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:42.852518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:42.852519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:42.852523 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:42.852537 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:42.852542 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:42.852594 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:42.852656 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 03:36:42.852666 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":19}
I20260809 03:36:42.854596 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 26193
W20260809 03:36:42.867311 26173 connection.cc:535] client connection to 127.0.0.1:34293 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34293): Connection reset by peer (error 104)
2026-08-09T03:36:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1507 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:42 dist-test-slave-gkw7 krb5kdc[26328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:42 dist-test-slave-gkw7 krb5kdc[26328](info): set up 2 sockets
Aug 09 03:36:42 dist-test-slave-gkw7 krb5kdc[26328](info): commencing operation
krb5kdc: starting...
W20260809 03:36:44.076349 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:44 dist-test-slave-gkw7 krb5kdc[26328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:36:44.196204 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45677
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:44.278481 26344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:44.278679 26344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:44.278702 26344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:44.279762 26344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:44.279799 26344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:44.279814 26344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:44.279825 26344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:44.279837 26344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:44.281291 26344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:44.281477 26344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:44.281692 26344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:44.286469 26351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:44.286509 26344 server_base.cc:1034] running on GCE node
W20260809 03:36:44.286535 26355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:44.286504 26350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:44.286909 26344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:44.287181 26344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:44.288340 26344 hybrid_clock.cc:648] HybridClock initialized: now 1786246604288322 us; error 45 us; skew 500 ppm
Aug 09 03:36:44 dist-test-slave-gkw7 krb5kdc[26328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246604, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:44.294354 26344 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:36:44.294832 26344 webserver.cc:460] Webserver started at http://127.0.0.1:41695/ using document root <none> and password file <none>
I20260809 03:36:44.295069 26344 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:44.295137 26344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:44.295275 26344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:44.296329 26344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "cd3c1d0ea0f14dd0833ad5d90a74fa7b"
format_stamp: "Formatted at 2026-08-09 03:36:44 on dist-test-slave-gkw7"
server_key: "c5926a9178d5fd2480ec05240cb017aa"
server_key_iv: "f38bbe6db35de107a52b3ca7d7c4d910"
server_key_version: "encryptionkey@0"
I20260809 03:36:44.296702 26344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "cd3c1d0ea0f14dd0833ad5d90a74fa7b"
format_stamp: "Formatted at 2026-08-09 03:36:44 on dist-test-slave-gkw7"
server_key: "c5926a9178d5fd2480ec05240cb017aa"
server_key_iv: "f38bbe6db35de107a52b3ca7d7c4d910"
server_key_version: "encryptionkey@0"
I20260809 03:36:44.298169 26344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260809 03:36:44.298898 26364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:44.299063 26344 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:36:44.299168 26344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "cd3c1d0ea0f14dd0833ad5d90a74fa7b"
format_stamp: "Formatted at 2026-08-09 03:36:44 on dist-test-slave-gkw7"
server_key: "c5926a9178d5fd2480ec05240cb017aa"
server_key_iv: "f38bbe6db35de107a52b3ca7d7c4d910"
server_key_version: "encryptionkey@0"
I20260809 03:36:44.299248 26344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:44.302631 26344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:44.304459 26344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:44.304611 26344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:44.308796 26344 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45677
I20260809 03:36:44.308837 26445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45677 every 8 connection(s)
I20260809 03:36:44.309180 26344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:44.310235 26446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:44.310842 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 26344
I20260809 03:36:44.310981 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:44.311177 27810 external_mini_cluster.cc:1428] Setting key efb840bb52ffd70eaac62f0e269a3d80
I20260809 03:36:44.313342 26446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b: Bootstrap starting.
I20260809 03:36:44.313633 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:44.311705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:51161 (local address 127.0.0.1:45677)
0809 03:36:44.311881 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:44.311888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:44.311913 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:36:44.312302 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:44.312308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:44.312332 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:44.312446 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:44.313069 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:44.313072 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:44.313072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:44.313076 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:44.313113 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:44.313123 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:44.313177 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:44.313270 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 03:36:44.313325 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260809 03:36:44.314139 26446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:44.314509 26446 log.cc:824] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:44.314576 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:44.315177 26446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b: No bootstrap required, opened a new log
I20260809 03:36:44.316993 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:44.316030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:58505 (local address 127.0.0.1:45677)
0809 03:36:44.316068 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:44.316069 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:44.316155 (+ 86us) server_negotiation.cc:407] Connection header received
0809 03:36:44.316217 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:44.316218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:44.316223 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:44.316265 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:44.316776 (+ 511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:44.316777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:44.316778 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:44.316781 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:44.316793 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:44.316796 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:44.316837 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:44.316902 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 03:36:44.316912 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 03:36:44.317605 26446 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd3c1d0ea0f14dd0833ad5d90a74fa7b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45677 } }
I20260809 03:36:44.317761 26446 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:44.317807 26446 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd3c1d0ea0f14dd0833ad5d90a74fa7b, State: Initialized, Role: FOLLOWER
I20260809 03:36:44.317929 26446 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd3c1d0ea0f14dd0833ad5d90a74fa7b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45677 } }
I20260809 03:36:44.318017 26446 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:44.318063 26446 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:44.318111 26446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:44.318635 26446 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd3c1d0ea0f14dd0833ad5d90a74fa7b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45677 } }
I20260809 03:36:44.318763 26446 leader_election.cc:302] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd3c1d0ea0f14dd0833ad5d90a74fa7b; no voters:
I20260809 03:36:44.318933 26446 leader_election.cc:288] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:44.318981 26458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:44.319178 26458 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [term 1 LEADER]: Becoming Leader. State: Replica: cd3c1d0ea0f14dd0833ad5d90a74fa7b, State: Running, Role: LEADER
I20260809 03:36:44.319181 26446 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:44.319294 26458 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd3c1d0ea0f14dd0833ad5d90a74fa7b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45677 } }
I20260809 03:36:44.319680 26460 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd3c1d0ea0f14dd0833ad5d90a74fa7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd3c1d0ea0f14dd0833ad5d90a74fa7b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45677 } } }
I20260809 03:36:44.319682 26463 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd3c1d0ea0f14dd0833ad5d90a74fa7b. Latest consensus state: current_term: 1 leader_uuid: "cd3c1d0ea0f14dd0833ad5d90a74fa7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd3c1d0ea0f14dd0833ad5d90a74fa7b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45677 } } }
I20260809 03:36:44.319868 26460 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:44.319928 26463 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:44.320279 26473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:44.320770 26473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:44.322400 26473 catalog_manager.cc:1347] Generated new cluster ID: faa4d920564f47f2bc24de36541c2956
I20260809 03:36:44.322458 26473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:44.334538 26473 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:44.334715 26473 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:44.340520 26473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd3c1d0ea0f14dd0833ad5d90a74fa7b: Generated new TSK 0
I20260809 03:36:44.340724 26473 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:44.343029 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 26344
2026-08-09T03:36:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1489 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:44 dist-test-slave-gkw7 krb5kdc[26490](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:44 dist-test-slave-gkw7 krb5kdc[26490](info): set up 2 sockets
Aug 09 03:36:44 dist-test-slave-gkw7 krb5kdc[26490](info): commencing operation
krb5kdc: starting...
W20260809 03:36:45.563206 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:45 dist-test-slave-gkw7 krb5kdc[26490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:36:45.687945 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42225
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:45.769104 26507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:45.769294 26507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:45.769318 26507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:45.770424 26507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:45.770463 26507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:45.770478 26507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:45.770490 26507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:45.770503 26507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:45.772078 26507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42225
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:45.772297 26507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:45.772523 26507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:45.777341 26520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:45.777377 26517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:45.777504 26507 server_base.cc:1034] running on GCE node
W20260809 03:36:45.777343 26518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:45.777828 26507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:45.778105 26507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:45.779278 26507 hybrid_clock.cc:648] HybridClock initialized: now 1786246605779262 us; error 36 us; skew 500 ppm
Aug 09 03:36:45 dist-test-slave-gkw7 krb5kdc[26490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246605, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:45.781730 26507 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:36:45.782140 26507 webserver.cc:460] Webserver started at http://127.0.0.1:37651/ using document root <none> and password file <none>
I20260809 03:36:45.782425 26507 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:45.782501 26507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:45.782637 26507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:45.783646 26507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "4240921055984ecc92b362132d50bc70"
format_stamp: "Formatted at 2026-08-09 03:36:45 on dist-test-slave-gkw7"
server_key: "a5cf43def010007f22aa1a8b0a01db7c"
server_key_iv: "9394097e881577ba81d93426b0108f02"
server_key_version: "encryptionkey@0"
I20260809 03:36:45.784018 26507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "4240921055984ecc92b362132d50bc70"
format_stamp: "Formatted at 2026-08-09 03:36:45 on dist-test-slave-gkw7"
server_key: "a5cf43def010007f22aa1a8b0a01db7c"
server_key_iv: "9394097e881577ba81d93426b0108f02"
server_key_version: "encryptionkey@0"
I20260809 03:36:45.785368 26507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:36:45.786188 26529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:45.786379 26507 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:36:45.786475 26507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "4240921055984ecc92b362132d50bc70"
format_stamp: "Formatted at 2026-08-09 03:36:45 on dist-test-slave-gkw7"
server_key: "a5cf43def010007f22aa1a8b0a01db7c"
server_key_iv: "9394097e881577ba81d93426b0108f02"
server_key_version: "encryptionkey@0"
I20260809 03:36:45.786558 26507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:45.816763 26507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:45.818985 26507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:45.819155 26507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:45.823251 26507 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42225
I20260809 03:36:45.823294 26606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42225 every 8 connection(s)
I20260809 03:36:45.823678 26507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:45.824643 26607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:45.827054 26607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70: Bootstrap starting.
I20260809 03:36:45.827617 26607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:45.827868 26607 log.cc:824] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:45.828539 26607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70: No bootstrap required, opened a new log
I20260809 03:36:45.830132 26607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4240921055984ecc92b362132d50bc70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42225 } }
I20260809 03:36:45.830271 26607 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:45.830293 26607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4240921055984ecc92b362132d50bc70, State: Initialized, Role: FOLLOWER
I20260809 03:36:45.830430 26607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4240921055984ecc92b362132d50bc70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42225 } }
I20260809 03:36:45.830513 26607 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:45.830536 26607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:45.830570 26607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:45.831053 26607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4240921055984ecc92b362132d50bc70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42225 } }
I20260809 03:36:45.831140 26607 leader_election.cc:302] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4240921055984ecc92b362132d50bc70; no voters:
I20260809 03:36:45.831262 26607 leader_election.cc:288] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:45.831322 26610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:45.831504 26610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [term 1 LEADER]: Becoming Leader. State: Replica: 4240921055984ecc92b362132d50bc70, State: Running, Role: LEADER
I20260809 03:36:45.831517 26607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:45.831678 26610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4240921055984ecc92b362132d50bc70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42225 } }
I20260809 03:36:45.832139 26611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4240921055984ecc92b362132d50bc70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4240921055984ecc92b362132d50bc70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42225 } } }
I20260809 03:36:45.832223 26611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:45.832273 26612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4240921055984ecc92b362132d50bc70. Latest consensus state: current_term: 1 leader_uuid: "4240921055984ecc92b362132d50bc70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4240921055984ecc92b362132d50bc70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42225 } } }
I20260809 03:36:45.832360 26612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:45.832617 26622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:45.833155 26622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:45.833187 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 26507
I20260809 03:36:45.833328 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:45.833559 27810 external_mini_cluster.cc:1428] Setting key 8fe569f4da3a2a55088030a1202bf156
I20260809 03:36:45.835143 26622 catalog_manager.cc:1347] Generated new cluster ID: 611f451dfa904d90a22f430630d1ae5c
I20260809 03:36:45.835198 26622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 03:36:45 dist-test-slave-gkw7 krb5kdc[26490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 03:36:45.845327 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:45.834018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:46603 (local address 127.0.0.1:42225)
0809 03:36:45.834190 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:45.834201 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:45.834239 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:36:45.834352 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:45.834359 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:45.834416 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:45.834562 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:45.835362 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:45.836784 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:45.837911 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:45.838546 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:45.838674 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:45.841604 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:45.841608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:45.841608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:45.841619 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:45.843586 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:45.844053 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:45.844056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:45.844056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:45.844157 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:45.844401 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:45.844401 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:45.844402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:45.844707 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:45.844856 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:45.845000 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 03:36:45.845093 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260809 03:36:45.846554 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:45.855229 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:45.847915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:56799 (local address 127.0.0.1:42225)
0809 03:36:45.847946 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:45.847948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:45.848054 (+ 106us) server_negotiation.cc:407] Connection header received
0809 03:36:45.848131 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:45.848132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:45.848140 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:45.848167 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:45.848843 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:45.849581 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:45.850645 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:45.850964 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:45.851020 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:45.852453 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:45.852454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:45.852455 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:45.852461 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:45.853997 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:45.854433 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:45.854434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:45.854434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:45.854490 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:45.854722 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:45.854723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:45.854723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:45.854836 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:45.854923 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:45.855075 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 03:36:45.855137 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 03:36:45.862478 26622 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:45.863184 26622 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:45.868323 26622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4240921055984ecc92b362132d50bc70: Generated new TSK 0
I20260809 03:36:45.868541 26622 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:45.879302 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 26507
W20260809 03:36:45.889493 26486 connection.cc:535] client connection to 127.0.0.1:42225 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42225): Connection reset by peer (error 104)
2026-08-09T03:36:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:45 dist-test-slave-gkw7 krb5kdc[26647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:45 dist-test-slave-gkw7 krb5kdc[26647](info): set up 2 sockets
Aug 09 03:36:45 dist-test-slave-gkw7 krb5kdc[26647](info): commencing operation
krb5kdc: starting...
W20260809 03:36:47.096478 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:47 dist-test-slave-gkw7 krb5kdc[26647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:36:47.220463 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44349
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:47.308167 26667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:47.308393 26667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:47.308429 26667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:47.309603 26667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:47.309665 26667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:47.309688 26667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:47.309715 26667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:47.309746 26667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:47.311228 26667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44349
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:47.311494 26667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:47.311714 26667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:47.316680 26683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:47.316619 26677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:47.316629 26676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:47.316983 26667 server_base.cc:1034] running on GCE node
I20260809 03:36:47.317281 26667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:47.317677 26667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:47.318842 26667 hybrid_clock.cc:648] HybridClock initialized: now 1786246607318830 us; error 31 us; skew 500 ppm
Aug 09 03:36:47 dist-test-slave-gkw7 krb5kdc[26647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246607, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:47.321614 26667 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:36:47.322082 26667 webserver.cc:460] Webserver started at http://127.0.0.1:40115/ using document root <none> and password file <none>
I20260809 03:36:47.322327 26667 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:47.322384 26667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:47.322562 26667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:47.323709 26667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "71f31e69b25b495ea52b35992b8e02d8"
format_stamp: "Formatted at 2026-08-09 03:36:47 on dist-test-slave-gkw7"
server_key: "445b54cbc2360e91c2050694fe26fe3f"
server_key_iv: "8705a5d038e3d2fb7dfcef2d8becd821"
server_key_version: "encryptionkey@0"
I20260809 03:36:47.324111 26667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "71f31e69b25b495ea52b35992b8e02d8"
format_stamp: "Formatted at 2026-08-09 03:36:47 on dist-test-slave-gkw7"
server_key: "445b54cbc2360e91c2050694fe26fe3f"
server_key_iv: "8705a5d038e3d2fb7dfcef2d8becd821"
server_key_version: "encryptionkey@0"
I20260809 03:36:47.325629 26667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:36:47.326417 26691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:47.326699 26667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:47.326799 26667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "71f31e69b25b495ea52b35992b8e02d8"
format_stamp: "Formatted at 2026-08-09 03:36:47 on dist-test-slave-gkw7"
server_key: "445b54cbc2360e91c2050694fe26fe3f"
server_key_iv: "8705a5d038e3d2fb7dfcef2d8becd821"
server_key_version: "encryptionkey@0"
I20260809 03:36:47.326869 26667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:47.343278 26667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:47.345829 26667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:47.345999 26667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:47.350394 26667 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44349
I20260809 03:36:47.350442 26761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44349 every 8 connection(s)
I20260809 03:36:47.350832 26667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:47.351748 26762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:47.354089 26762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8: Bootstrap starting.
I20260809 03:36:47.354719 26762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:47.355013 26762 log.cc:824] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:47.355772 26762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8: No bootstrap required, opened a new log
I20260809 03:36:47.356002 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 26667
I20260809 03:36:47.356181 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:47.356381 27810 external_mini_cluster.cc:1428] Setting key 6e717ee1e81c24bbe82f2cbed40cd415
I20260809 03:36:47.358108 26762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f31e69b25b495ea52b35992b8e02d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } }
I20260809 03:36:47.358319 26762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:47.358357 26762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71f31e69b25b495ea52b35992b8e02d8, State: Initialized, Role: FOLLOWER
I20260809 03:36:47.358487 26762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f31e69b25b495ea52b35992b8e02d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } }
I20260809 03:36:47.358578 26762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:47.358654 26762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:47.358707 26762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:47.359272 26762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f31e69b25b495ea52b35992b8e02d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } }
I20260809 03:36:47.359392 26762 leader_election.cc:302] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71f31e69b25b495ea52b35992b8e02d8; no voters:
I20260809 03:36:47.359548 26762 leader_election.cc:288] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:47.359588 26767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:47.359760 26767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [term 1 LEADER]: Becoming Leader. State: Replica: 71f31e69b25b495ea52b35992b8e02d8, State: Running, Role: LEADER
I20260809 03:36:47.359831 26762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:47.359906 26767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f31e69b25b495ea52b35992b8e02d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } }
I20260809 03:36:47.360235 26767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71f31e69b25b495ea52b35992b8e02d8. Latest consensus state: current_term: 1 leader_uuid: "71f31e69b25b495ea52b35992b8e02d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f31e69b25b495ea52b35992b8e02d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } } }
I20260809 03:36:47.360349 26767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:47.360630 26780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:47.361207 26780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:47.363232 26772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71f31e69b25b495ea52b35992b8e02d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f31e69b25b495ea52b35992b8e02d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } } }
I20260809 03:36:47.363324 26772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:47.363994 26766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:47.356931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:48211 (local address 127.0.0.1:44349)
0809 03:36:47.357136 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:47.357145 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:47.357180 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:36:47.358337 (+ 1157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:47.358347 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:47.358413 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:47.358563 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:47.358840 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:47.360068 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:47.361243 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:47.362233 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:47.362348 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:47.363198 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:47.363203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:47.363203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:47.363456 (+ 253us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:47.363501 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:47.363514 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:47.363613 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:47.363706 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 03:36:47.363736 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":89,"threads_started":1}
I20260809 03:36:47.364354 26780 catalog_manager.cc:1347] Generated new cluster ID: 6f47d16f4cae47fea01d6c97aee447fb
I20260809 03:36:47.364459 26780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:47.365120 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:47.370054 26766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:47.366684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:54001 (local address 127.0.0.1:44349)
0809 03:36:47.366721 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:47.366724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:47.366938 (+ 214us) server_negotiation.cc:407] Connection header received
0809 03:36:47.367022 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:47.367023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:47.367034 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:47.367096 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:47.367257 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:47.367761 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:47.368562 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:47.368873 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:47.368961 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:47.369756 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:47.369758 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:47.369758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:47.369763 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:47.369777 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:47.369782 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:47.369861 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:47.369941 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 03:36:47.369951 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 03:36:47.386991 26780 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:47.388317 26780 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:47.391168 26780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71f31e69b25b495ea52b35992b8e02d8: Generated new TSK 0
I20260809 03:36:47.391364 26780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:47.393632 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 26667
W20260809 03:36:47.405712 26644 connection.cc:535] client connection to 127.0.0.1:44349 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44349): Connection reset by peer (error 104)
2026-08-09T03:36:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:47 dist-test-slave-gkw7 krb5kdc[26813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:47 dist-test-slave-gkw7 krb5kdc[26813](info): set up 2 sockets
Aug 09 03:36:47 dist-test-slave-gkw7 krb5kdc[26813](info): commencing operation
krb5kdc: starting...
W20260809 03:36:48.633808 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:48 dist-test-slave-gkw7 krb5kdc[26813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:36:48.756865 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41703
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:48.839005 26830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:48.839253 26830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:48.839293 26830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:48.840417 26830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:48.840485 26830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:48.840510 26830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:48.840539 26830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:48.840569 26830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:48.842073 26830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41703
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:48.842338 26830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:48.842564 26830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:48.847852 26844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:48.848091 26830 server_base.cc:1034] running on GCE node
W20260809 03:36:48.847870 26842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:48.847857 26841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:48.848440 26830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:48.848774 26830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:48.849927 26830 hybrid_clock.cc:648] HybridClock initialized: now 1786246608849910 us; error 38 us; skew 500 ppm
Aug 09 03:36:48 dist-test-slave-gkw7 krb5kdc[26813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246608, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:48.852450 26830 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:36:48.852888 26830 webserver.cc:460] Webserver started at http://127.0.0.1:36391/ using document root <none> and password file <none>
I20260809 03:36:48.853096 26830 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:48.853137 26830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:48.853227 26830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:48.854269 26830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "09c222245f83458a84c16ca84b97da48"
format_stamp: "Formatted at 2026-08-09 03:36:48 on dist-test-slave-gkw7"
server_key: "a4dc20631487e1bd052fd1b06eb7d508"
server_key_iv: "16596794e2fe9a0a646800e862d88322"
server_key_version: "encryptionkey@0"
I20260809 03:36:48.854578 26830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "09c222245f83458a84c16ca84b97da48"
format_stamp: "Formatted at 2026-08-09 03:36:48 on dist-test-slave-gkw7"
server_key: "a4dc20631487e1bd052fd1b06eb7d508"
server_key_iv: "16596794e2fe9a0a646800e862d88322"
server_key_version: "encryptionkey@0"
I20260809 03:36:48.855875 26830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:36:48.856590 26853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:48.856776 26830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:48.856868 26830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "09c222245f83458a84c16ca84b97da48"
format_stamp: "Formatted at 2026-08-09 03:36:48 on dist-test-slave-gkw7"
server_key: "a4dc20631487e1bd052fd1b06eb7d508"
server_key_iv: "16596794e2fe9a0a646800e862d88322"
server_key_version: "encryptionkey@0"
I20260809 03:36:48.856922 26830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:48.860204 26830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:48.862222 26830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:48.862375 26830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:48.866240 26830 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41703
I20260809 03:36:48.866297 26924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41703 every 8 connection(s)
I20260809 03:36:48.866619 26830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:48.867585 26925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:48.869849 26925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48: Bootstrap starting.
I20260809 03:36:48.870376 26925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:48.870623 26925 log.cc:824] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:48.871325 26925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48: No bootstrap required, opened a new log
I20260809 03:36:48.872115 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 26830
I20260809 03:36:48.872304 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:48.872519 27810 external_mini_cluster.cc:1428] Setting key 8ef60a493eadcb972f05fb9a449dff22
I20260809 03:36:48.873298 26925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09c222245f83458a84c16ca84b97da48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41703 } }
I20260809 03:36:48.873490 26925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:48.873548 26925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09c222245f83458a84c16ca84b97da48, State: Initialized, Role: FOLLOWER
I20260809 03:36:48.873684 26925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09c222245f83458a84c16ca84b97da48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41703 } }
I20260809 03:36:48.873787 26925 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:48.873826 26925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:48.873883 26925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:48.874676 26925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09c222245f83458a84c16ca84b97da48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41703 } }
I20260809 03:36:48.874815 26925 leader_election.cc:302] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09c222245f83458a84c16ca84b97da48; no voters:
I20260809 03:36:48.874969 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:48.873036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:35115 (local address 127.0.0.1:41703)
0809 03:36:48.873298 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:48.873325 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:48.873360 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:36:48.873465 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:48.873471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:48.873502 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:48.873960 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:48.874496 (+ 536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:48.874500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:48.874500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:48.874505 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:48.874542 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:48.874557 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:48.874635 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:48.874695 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 03:36:48.874746 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":101,"threads_started":1}
I20260809 03:36:48.875000 26925 leader_election.cc:288] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:48.875073 26931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:48.875239 26925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:48.875335 26931 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [term 1 LEADER]: Becoming Leader. State: Replica: 09c222245f83458a84c16ca84b97da48, State: Running, Role: LEADER
I20260809 03:36:48.875440 26931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09c222245f83458a84c16ca84b97da48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41703 } }
I20260809 03:36:48.875862 26931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09c222245f83458a84c16ca84b97da48. Latest consensus state: current_term: 1 leader_uuid: "09c222245f83458a84c16ca84b97da48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09c222245f83458a84c16ca84b97da48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41703 } } }
I20260809 03:36:48.875918 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:48.876008 26931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:48.875955 26932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09c222245f83458a84c16ca84b97da48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09c222245f83458a84c16ca84b97da48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41703 } } }
I20260809 03:36:48.876231 26932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:48.876385 26944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:48.877351 26944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:48.879129 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:48.877722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:58613 (local address 127.0.0.1:41703)
0809 03:36:48.877757 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:48.877759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:48.877984 (+ 225us) server_negotiation.cc:407] Connection header received
0809 03:36:48.878096 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:48.878097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:48.878102 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:48.878136 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:48.878856 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:48.878857 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:48.878858 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:48.878862 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:48.878876 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:48.878880 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:48.878918 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:48.879052 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 03:36:48.879068 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 03:36:48.879382 26944 catalog_manager.cc:1347] Generated new cluster ID: 86cc990200ff451bb3e6d82d07ef9b76
I20260809 03:36:48.879443 26944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:48.886168 26944 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:48.886329 26944 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:48.891505 26944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09c222245f83458a84c16ca84b97da48: Generated new TSK 0
I20260809 03:36:48.891665 26944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:48.894541 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 26830
2026-08-09T03:36:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1501 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:48 dist-test-slave-gkw7 krb5kdc[26964](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:48 dist-test-slave-gkw7 krb5kdc[26964](info): set up 2 sockets
Aug 09 03:36:48 dist-test-slave-gkw7 krb5kdc[26964](info): commencing operation
krb5kdc: starting...
W20260809 03:36:50.124740 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:50 dist-test-slave-gkw7 krb5kdc[26964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:36:50.254475 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:46473
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:35349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:46473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:50.338616 26986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:50.338804 26986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:50.338826 26986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:50.339926 26986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:50.339964 26986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:50.339978 26986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:50.339991 26986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:50.340003 26986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:50.341444 26986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:35349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:46473
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:46473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:50.341686 26986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:50.341881 26986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:50.346730 26992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:50.346719 26993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:50.346796 26998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:50.346833 26986 server_base.cc:1034] running on GCE node
I20260809 03:36:50.347276 26986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:50.347606 26986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:50.348779 26986 hybrid_clock.cc:648] HybridClock initialized: now 1786246610348761 us; error 29 us; skew 500 ppm
Aug 09 03:36:50 dist-test-slave-gkw7 krb5kdc[26964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246610, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:50.355762 26986 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:36:50.356196 26986 webserver.cc:460] Webserver started at http://127.27.40.190:43711/ using document root <none> and password file <none>
I20260809 03:36:50.356446 26986 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:50.356516 26986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:50.356668 26986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:50.357779 26986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "762876c776b24ed6a88cec0b769b9ea0"
format_stamp: "Formatted at 2026-08-09 03:36:50 on dist-test-slave-gkw7"
server_key: "e71eefcbf3cf97dd01d14fb481ed9e4b"
server_key_iv: "8eb9d47e37ad0e25e305820464c14a6f"
server_key_version: "encryptionkey@0"
I20260809 03:36:50.358129 26986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "762876c776b24ed6a88cec0b769b9ea0"
format_stamp: "Formatted at 2026-08-09 03:36:50 on dist-test-slave-gkw7"
server_key: "e71eefcbf3cf97dd01d14fb481ed9e4b"
server_key_iv: "8eb9d47e37ad0e25e305820464c14a6f"
server_key_version: "encryptionkey@0"
I20260809 03:36:50.359550 26986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:36:50.360404 27008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:50.360591 26986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:50.360713 26986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "762876c776b24ed6a88cec0b769b9ea0"
format_stamp: "Formatted at 2026-08-09 03:36:50 on dist-test-slave-gkw7"
server_key: "e71eefcbf3cf97dd01d14fb481ed9e4b"
server_key_iv: "8eb9d47e37ad0e25e305820464c14a6f"
server_key_version: "encryptionkey@0"
I20260809 03:36:50.360791 26986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:50.371729 26986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:50.373427 26986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:50.373607 26986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:50.377873 26986 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:46473
I20260809 03:36:50.377904 27084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:46473 every 8 connection(s)
I20260809 03:36:50.378258 26986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:50.379112 27085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:50.379817 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 26986
I20260809 03:36:50.379942 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:50.380127 27810 external_mini_cluster.cc:1428] Setting key cd34c5e1d9e5bdf72bfb659eabc7b461
I20260809 03:36:50.382289 27085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0: Bootstrap starting.
I20260809 03:36:50.382923 27085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:50.383219 27085 log.cc:824] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:50.383999 27085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0: No bootstrap required, opened a new log
I20260809 03:36:50.386556 27085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "762876c776b24ed6a88cec0b769b9ea0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 46473 } }
I20260809 03:36:50.386749 27085 raft_consensus.cc:374] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:50.386780 27085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 762876c776b24ed6a88cec0b769b9ea0, State: Initialized, Role: FOLLOWER
I20260809 03:36:50.386914 27085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "762876c776b24ed6a88cec0b769b9ea0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 46473 } }
I20260809 03:36:50.386987 27085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:50.387009 27085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:50.387068 27085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 03:36:50 dist-test-slave-gkw7 krb5kdc[26964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:50.387646 27085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "762876c776b24ed6a88cec0b769b9ea0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 46473 } }
I20260809 03:36:50.387771 27085 leader_election.cc:302] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 762876c776b24ed6a88cec0b769b9ea0; no voters:
I20260809 03:36:50.387991 27085 leader_election.cc:288] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:50.388029 27091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:50.388146 27091 raft_consensus.cc:686] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [term 1 LEADER]: Becoming Leader. State: Replica: 762876c776b24ed6a88cec0b769b9ea0, State: Running, Role: LEADER
I20260809 03:36:50.388279 27085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:50.388300 27091 consensus_queue.cc:237] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "762876c776b24ed6a88cec0b769b9ea0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 46473 } }
I20260809 03:36:50.388710 27093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "762876c776b24ed6a88cec0b769b9ea0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "762876c776b24ed6a88cec0b769b9ea0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 46473 } } }
I20260809 03:36:50.388800 27093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:50.389016 27096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 762876c776b24ed6a88cec0b769b9ea0. Latest consensus state: current_term: 1 leader_uuid: "762876c776b24ed6a88cec0b769b9ea0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "762876c776b24ed6a88cec0b769b9ea0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 46473 } } }
I20260809 03:36:50.389153 27096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0 [sys.catalog]: This master's current role is: LEADER
W20260809 03:36:50.390269 27115 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:36:50.390362 27115 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:36:50.390440 27101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:50.390586 27101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:50.391712 27088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:50.380673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60301 (local address 127.27.40.190:46473)
0809 03:36:50.380853 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:50.380860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:50.380885 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:36:50.381301 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:50.381307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:50.381353 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:50.381465 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:50.382392 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.383356 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.384367 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.384796 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.384881 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:50.387789 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:50.387793 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:50.387793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:50.387805 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:50.390011 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:50.390468 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:50.390471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:50.390472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:50.390577 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:50.390848 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:50.390849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:50.390849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:50.391095 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:50.391256 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:50.391391 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 03:36:50.391495 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260809 03:36:50.392740 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:50.394014 27101 catalog_manager.cc:1347] Generated new cluster ID: 7c6d6243522743908319952d53976c29
I20260809 03:36:50.394081 27101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:50.404853 27088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:50.397665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52081 (local address 127.27.40.190:46473)
0809 03:36:50.397704 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:50.397706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:50.397714 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:36:50.397733 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:50.397734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:50.397743 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:50.397801 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:50.398481 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.399225 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.400252 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.400567 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.400625 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:50.402114 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:50.402116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:50.402116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:50.402123 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:50.403645 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:50.404063 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:50.404065 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:50.404065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:50.404122 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:50.404356 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:50.404357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:50.404357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:50.404472 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:50.404585 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:50.404708 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 03:36:50.404766 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:36:50.406847 27101 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:50.407543 27101 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:50.414860 27101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 762876c776b24ed6a88cec0b769b9ea0: Generated new TSK 0
I20260809 03:36:50.415024 27101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:50.421020 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 26986
W20260809 03:36:50.434520 26962 connection.cc:535] client connection to 127.27.40.190:46473 recv error: Network error: failed to read from TLS socket (remote: 127.27.40.190:46473): Connection reset by peer (error 104)
2026-08-09T03:36:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1527 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:50 dist-test-slave-gkw7 krb5kdc[27128](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:50 dist-test-slave-gkw7 krb5kdc[27128](info): set up 2 sockets
Aug 09 03:36:50 dist-test-slave-gkw7 krb5kdc[27128](info): commencing operation
krb5kdc: starting...
W20260809 03:36:51.654146 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:51 dist-test-slave-gkw7 krb5kdc[27128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:36:51.779974 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:35981
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:41745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:35981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:51.883150 27146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:51.883345 27146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:51.883368 27146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:51.884493 27146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:51.884536 27146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:51.884550 27146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:51.884563 27146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:51.884617 27146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:51.886122 27146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:41745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:35981
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:35981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:51.886363 27146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:51.886586 27146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:51.891594 27152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:51.891603 27151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:51.891629 27156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:51.891817 27146 server_base.cc:1034] running on GCE node
I20260809 03:36:51.892079 27146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:51.892376 27146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:51.893590 27146 hybrid_clock.cc:648] HybridClock initialized: now 1786246611893562 us; error 40 us; skew 500 ppm
Aug 09 03:36:51 dist-test-slave-gkw7 krb5kdc[27128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246611, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:51.896245 27146 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:36:51.896724 27146 webserver.cc:460] Webserver started at http://127.27.40.190:44793/ using document root <none> and password file <none>
I20260809 03:36:51.896977 27146 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:51.897046 27146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:51.897200 27146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:51.898306 27146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "30f3f51052b84dcfb46e3e8e9b9cb25e"
format_stamp: "Formatted at 2026-08-09 03:36:51 on dist-test-slave-gkw7"
server_key: "3ddfccfe4859fa58a2153647127e1aff"
server_key_iv: "e1667471088944a14d8a4bfaa9fcc8a0"
server_key_version: "encryptionkey@0"
I20260809 03:36:51.898689 27146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "30f3f51052b84dcfb46e3e8e9b9cb25e"
format_stamp: "Formatted at 2026-08-09 03:36:51 on dist-test-slave-gkw7"
server_key: "3ddfccfe4859fa58a2153647127e1aff"
server_key_iv: "e1667471088944a14d8a4bfaa9fcc8a0"
server_key_version: "encryptionkey@0"
I20260809 03:36:51.900056 27146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 03:36:51.900822 27173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:51.901000 27146 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:36:51.901090 27146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "30f3f51052b84dcfb46e3e8e9b9cb25e"
format_stamp: "Formatted at 2026-08-09 03:36:51 on dist-test-slave-gkw7"
server_key: "3ddfccfe4859fa58a2153647127e1aff"
server_key_iv: "e1667471088944a14d8a4bfaa9fcc8a0"
server_key_version: "encryptionkey@0"
I20260809 03:36:51.901173 27146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:51.915236 27146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:51.917739 27146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:51.917910 27146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:51.922403 27146 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:35981
I20260809 03:36:51.922441 27247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:35981 every 8 connection(s)
I20260809 03:36:51.922974 27146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:51.923738 27248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:51.926234 27248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e: Bootstrap starting.
I20260809 03:36:51.926683 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 27146
I20260809 03:36:51.926877 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:51.926949 27248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:51.927093 27810 external_mini_cluster.cc:1428] Setting key 17f5e6d46273d072883f1c6d385430d5
I20260809 03:36:51.927299 27248 log.cc:824] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:51.928138 27248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e: No bootstrap required, opened a new log
I20260809 03:36:51.930867 27248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f3f51052b84dcfb46e3e8e9b9cb25e" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35981 } }
I20260809 03:36:51.931020 27248 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:51.931042 27248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f3f51052b84dcfb46e3e8e9b9cb25e, State: Initialized, Role: FOLLOWER
I20260809 03:36:51.931191 27248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f3f51052b84dcfb46e3e8e9b9cb25e" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35981 } }
I20260809 03:36:51.931270 27248 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:51.931315 27248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:51.931367 27248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:51.932181 27248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f3f51052b84dcfb46e3e8e9b9cb25e" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35981 } }
I20260809 03:36:51.932304 27248 leader_election.cc:302] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30f3f51052b84dcfb46e3e8e9b9cb25e; no voters:
I20260809 03:36:51.932497 27248 leader_election.cc:288] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:51.932562 27256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:51.932785 27256 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [term 1 LEADER]: Becoming Leader. State: Replica: 30f3f51052b84dcfb46e3e8e9b9cb25e, State: Running, Role: LEADER
I20260809 03:36:51.932901 27248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:51.932960 27256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f3f51052b84dcfb46e3e8e9b9cb25e" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35981 } }
I20260809 03:36:51.933983 27257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30f3f51052b84dcfb46e3e8e9b9cb25e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f3f51052b84dcfb46e3e8e9b9cb25e" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35981 } } }
I20260809 03:36:51.934083 27257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:51.934010 27258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30f3f51052b84dcfb46e3e8e9b9cb25e. Latest consensus state: current_term: 1 leader_uuid: "30f3f51052b84dcfb46e3e8e9b9cb25e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f3f51052b84dcfb46e3e8e9b9cb25e" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35981 } } }
I20260809 03:36:51.934155 27258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:51.934356 27267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:51.934857 27254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:51.927687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42701 (local address 127.27.40.190:35981)
0809 03:36:51.927838 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:51.927847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:51.927906 (+ 59us) server_negotiation.cc:407] Connection header received
0809 03:36:51.928534 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:51.928543 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:51.928606 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:51.928751 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:51.928926 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:51.930466 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:51.931611 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:51.932276 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:51.932447 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:51.933719 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:51.933725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:51.933726 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:51.934034 (+ 308us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:51.934087 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:51.934107 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:51.934237 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:51.934582 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 03:36:51.934621 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":65,"threads_started":1}
I20260809 03:36:51.935062 27267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:51.936031 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:51.936887 27267 catalog_manager.cc:1347] Generated new cluster ID: 98b892b62d164c1f9906f46ed072653a
I20260809 03:36:51.936926 27267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:51.941658 27254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:51.937677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53001 (local address 127.27.40.190:35981)
0809 03:36:51.937722 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:51.937724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:51.937894 (+ 170us) server_negotiation.cc:407] Connection header received
0809 03:36:51.937943 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:51.937944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:51.937955 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:51.938013 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:51.938160 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:51.938994 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:51.940074 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:51.940426 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:51.940490 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:51.941285 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:51.941286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:51.941286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:51.941292 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:51.941337 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:51.941341 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:51.941406 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:51.941554 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 03:36:51.941568 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 03:36:51.962574 27267 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:51.963255 27267 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:51.967419 27267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30f3f51052b84dcfb46e3e8e9b9cb25e: Generated new TSK 0
I20260809 03:36:51.967653 27267 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:51.974637 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 27146
W20260809 03:36:51.986953 27125 connection.cc:535] client connection to 127.27.40.190:35981 recv error: Network error: failed to read from TLS socket (remote: 127.27.40.190:35981): Connection reset by peer (error 104)
2026-08-09T03:36:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1552 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:52 dist-test-slave-gkw7 krb5kdc[27291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:52 dist-test-slave-gkw7 krb5kdc[27291](info): set up 2 sockets
Aug 09 03:36:52 dist-test-slave-gkw7 krb5kdc[27291](info): commencing operation
krb5kdc: starting...
W20260809 03:36:53.249893 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:53 dist-test-slave-gkw7 krb5kdc[27291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:36:53.380844 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:42435
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:42435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:53.468374 27315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:53.468645 27315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:53.468688 27315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:53.469898 27315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:53.469975 27315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:53.470003 27315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:53.470029 27315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:53.470065 27315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:53.471614 27315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:42435
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:42435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:53.471903 27315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:53.472149 27315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:53.477195 27323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:53.477178 27325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:53.477320 27315 server_base.cc:1034] running on GCE node
W20260809 03:36:53.477176 27327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:53.477703 27315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:53.478041 27315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:53.479198 27315 hybrid_clock.cc:648] HybridClock initialized: now 1786246613479176 us; error 42 us; skew 500 ppm
Aug 09 03:36:53 dist-test-slave-gkw7 krb5kdc[27291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246613, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:53.481762 27315 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:36:53.482232 27315 webserver.cc:460] Webserver started at http://127.27.40.190:44497/ using document root <none> and password file <none>
I20260809 03:36:53.482415 27315 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:53.482455 27315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:53.482543 27315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:53.483707 27315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "7b77f21d3da34a7d802c215a4e2d2dd0"
format_stamp: "Formatted at 2026-08-09 03:36:53 on dist-test-slave-gkw7"
server_key: "962d6d66e190bddc763f9584c101529c"
server_key_iv: "8a3ae7df9a9146db3bdb5c6f8c7cf192"
server_key_version: "encryptionkey@0"
I20260809 03:36:53.484018 27315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "7b77f21d3da34a7d802c215a4e2d2dd0"
format_stamp: "Formatted at 2026-08-09 03:36:53 on dist-test-slave-gkw7"
server_key: "962d6d66e190bddc763f9584c101529c"
server_key_iv: "8a3ae7df9a9146db3bdb5c6f8c7cf192"
server_key_version: "encryptionkey@0"
I20260809 03:36:53.485345 27315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:36:53.486217 27338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:53.486392 27315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 03:36:53.486490 27315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "7b77f21d3da34a7d802c215a4e2d2dd0"
format_stamp: "Formatted at 2026-08-09 03:36:53 on dist-test-slave-gkw7"
server_key: "962d6d66e190bddc763f9584c101529c"
server_key_iv: "8a3ae7df9a9146db3bdb5c6f8c7cf192"
server_key_version: "encryptionkey@0"
I20260809 03:36:53.486548 27315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:53.489830 27315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:53.492156 27315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:53.492287 27315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:53.496209 27315 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:42435
I20260809 03:36:53.496256 27411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:42435 every 8 connection(s)
I20260809 03:36:53.496635 27315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:53.497699 27412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:53.499944 27412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0: Bootstrap starting.
I20260809 03:36:53.500527 27412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:53.500824 27412 log.cc:824] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:53.501680 27412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0: No bootstrap required, opened a new log
I20260809 03:36:53.503312 27412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b77f21d3da34a7d802c215a4e2d2dd0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 42435 } }
I20260809 03:36:53.503465 27412 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:53.503510 27412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b77f21d3da34a7d802c215a4e2d2dd0, State: Initialized, Role: FOLLOWER
I20260809 03:36:53.503679 27412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b77f21d3da34a7d802c215a4e2d2dd0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 42435 } }
I20260809 03:36:53.503760 27412 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:53.503799 27412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:53.503851 27412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:53.504688 27412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b77f21d3da34a7d802c215a4e2d2dd0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 42435 } }
I20260809 03:36:53.504809 27412 leader_election.cc:302] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b77f21d3da34a7d802c215a4e2d2dd0; no voters:
I20260809 03:36:53.504992 27412 leader_election.cc:288] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:53.505024 27418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:53.505204 27418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [term 1 LEADER]: Becoming Leader. State: Replica: 7b77f21d3da34a7d802c215a4e2d2dd0, State: Running, Role: LEADER
I20260809 03:36:53.505261 27412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:53.505367 27418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b77f21d3da34a7d802c215a4e2d2dd0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 42435 } }
I20260809 03:36:53.505760 27421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b77f21d3da34a7d802c215a4e2d2dd0. Latest consensus state: current_term: 1 leader_uuid: "7b77f21d3da34a7d802c215a4e2d2dd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b77f21d3da34a7d802c215a4e2d2dd0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 42435 } } }
I20260809 03:36:53.505749 27419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b77f21d3da34a7d802c215a4e2d2dd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b77f21d3da34a7d802c215a4e2d2dd0" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 42435 } } }
I20260809 03:36:53.505853 27421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:53.505861 27419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:53.506083 27429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:53.506577 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 27315
I20260809 03:36:53.506699 27429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:53.506708 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:53.506896 27810 external_mini_cluster.cc:1428] Setting key bc07474ccbba97f65c15bfaeeb2b78b6
I20260809 03:36:53.509121 27429 catalog_manager.cc:1347] Generated new cluster ID: a6438fa4f7ae4481a8bd58bd4e3f1ea4
I20260809 03:36:53.509176 27429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:53.509404 27441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:53.507473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50819 (local address 127.27.40.190:42435)
0809 03:36:53.507789 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:53.507798 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:53.507832 (+ 34us) server_negotiation.cc:407] Connection header received
0809 03:36:53.507946 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:53.507953 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:53.507982 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:53.508391 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:53.508848 (+ 457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:53.508851 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:53.508851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:53.508856 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:53.508906 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:53.508921 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:53.509160 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:53.509191 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 03:36:53.509235 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":68,"threads_started":1}
I20260809 03:36:53.510309 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:53.512768 27441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:53.511772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33495 (local address 127.27.40.190:42435)
0809 03:36:53.511802 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:53.511804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:53.511917 (+ 113us) server_negotiation.cc:407] Connection header received
0809 03:36:53.511963 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:53.511964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:53.511969 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:53.512025 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:53.512574 (+ 549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:53.512575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:53.512575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:53.512579 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:53.512592 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:53.512597 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:53.512646 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:53.512701 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 03:36:53.512712 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 03:36:53.523665 27429 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:53.523859 27429 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:53.534799 27429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b77f21d3da34a7d802c215a4e2d2dd0: Generated new TSK 0
I20260809 03:36:53.535061 27429 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:53.544612 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 27315
2026-08-09T03:36:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1569 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:53 dist-test-slave-gkw7 krb5kdc[27451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:53 dist-test-slave-gkw7 krb5kdc[27451](info): set up 2 sockets
Aug 09 03:36:53 dist-test-slave-gkw7 krb5kdc[27451](info): commencing operation
krb5kdc: starting...
W20260809 03:36:54.774739 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:54 dist-test-slave-gkw7 krb5kdc[27451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:36:54.900254 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:34615
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:42287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:34615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:54.983116 27474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:54.983359 27474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:54.983397 27474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:54.984553 27474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:54.984633 27474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:54.984659 27474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:54.984688 27474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:54.984719 27474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:54.986271 27474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:42287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:34615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:34615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:54.986560 27474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:54.986843 27474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:54.992022 27480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:54.992054 27481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:54.992049 27484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:54.992331 27474 server_base.cc:1034] running on GCE node
I20260809 03:36:54.992550 27474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:54.992847 27474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:54.994015 27474 hybrid_clock.cc:648] HybridClock initialized: now 1786246614993996 us; error 35 us; skew 500 ppm
Aug 09 03:36:54 dist-test-slave-gkw7 krb5kdc[27451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246614, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:54.996580 27474 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:36:54.997097 27474 webserver.cc:460] Webserver started at http://127.27.40.190:35489/ using document root <none> and password file <none>
I20260809 03:36:54.997368 27474 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:54.997419 27474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:54.997637 27474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:54.998756 27474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "a76612802d294da095fba5d05b247b90"
format_stamp: "Formatted at 2026-08-09 03:36:54 on dist-test-slave-gkw7"
server_key: "5e71a9ff97d229bc995a1b1a7b252663"
server_key_iv: "7f5c536c123a57d8c7297f276c780ac4"
server_key_version: "encryptionkey@0"
I20260809 03:36:54.999150 27474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "a76612802d294da095fba5d05b247b90"
format_stamp: "Formatted at 2026-08-09 03:36:54 on dist-test-slave-gkw7"
server_key: "5e71a9ff97d229bc995a1b1a7b252663"
server_key_iv: "7f5c536c123a57d8c7297f276c780ac4"
server_key_version: "encryptionkey@0"
I20260809 03:36:55.000640 27474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:36:55.001457 27492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:55.001682 27474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:55.001765 27474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "a76612802d294da095fba5d05b247b90"
format_stamp: "Formatted at 2026-08-09 03:36:54 on dist-test-slave-gkw7"
server_key: "5e71a9ff97d229bc995a1b1a7b252663"
server_key_iv: "7f5c536c123a57d8c7297f276c780ac4"
server_key_version: "encryptionkey@0"
I20260809 03:36:55.001834 27474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:55.018829 27474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:55.020764 27474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:55.020884 27474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:55.025058 27474 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:34615
I20260809 03:36:55.025110 27563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:34615 every 8 connection(s)
I20260809 03:36:55.025497 27474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:55.026439 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:55.028734 27564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90: Bootstrap starting.
I20260809 03:36:55.029356 27564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:55.029697 27564 log.cc:824] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:55.030447 27564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90: No bootstrap required, opened a new log
I20260809 03:36:55.032001 27564 raft_consensus.cc:348] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76612802d294da095fba5d05b247b90" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34615 } }
I20260809 03:36:55.032135 27564 raft_consensus.cc:374] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:55.032157 27564 raft_consensus.cc:729] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a76612802d294da095fba5d05b247b90, State: Initialized, Role: FOLLOWER
I20260809 03:36:55.032336 27564 consensus_queue.cc:260] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76612802d294da095fba5d05b247b90" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34615 } }
I20260809 03:36:55.032400 27564 raft_consensus.cc:388] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:55.032450 27564 raft_consensus.cc:482] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:55.032516 27564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:55.033368 27564 raft_consensus.cc:504] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76612802d294da095fba5d05b247b90" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34615 } }
I20260809 03:36:55.033456 27564 leader_election.cc:302] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a76612802d294da095fba5d05b247b90; no voters:
I20260809 03:36:55.033671 27564 leader_election.cc:288] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:55.033725 27567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:55.033946 27567 raft_consensus.cc:686] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [term 1 LEADER]: Becoming Leader. State: Replica: a76612802d294da095fba5d05b247b90, State: Running, Role: LEADER
I20260809 03:36:55.033949 27564 sys_catalog.cc:564] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:55.034077 27567 consensus_queue.cc:237] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76612802d294da095fba5d05b247b90" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34615 } }
I20260809 03:36:55.034483 27568 sys_catalog.cc:455] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a76612802d294da095fba5d05b247b90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76612802d294da095fba5d05b247b90" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34615 } } }
I20260809 03:36:55.034596 27568 sys_catalog.cc:458] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:55.034483 27570 sys_catalog.cc:455] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a76612802d294da095fba5d05b247b90. Latest consensus state: current_term: 1 leader_uuid: "a76612802d294da095fba5d05b247b90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76612802d294da095fba5d05b247b90" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 34615 } } }
I20260809 03:36:55.034755 27570 sys_catalog.cc:458] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:55.034862 27579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:55.035491 27579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:55.035681 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 27474
I20260809 03:36:55.035804 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:55.035993 27810 external_mini_cluster.cc:1428] Setting key 745b83d5bdf80396b3703130510f0c49
I20260809 03:36:55.038074 27579 catalog_manager.cc:1347] Generated new cluster ID: ad30d414768f4f0caba450c7afa8bf31
I20260809 03:36:55.038139 27579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:55.043087 27579 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:55.043754 27579 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 03:36:55 dist-test-slave-gkw7 krb5kdc[27451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:55.050204 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:55.036736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42943 (local address 127.27.40.190:34615)
0809 03:36:55.036880 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:55.036889 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:55.036925 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:36:55.037048 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:55.037054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:55.037122 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:55.037277 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:55.041639 (+ 4362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:55.042590 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:55.043521 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:55.044041 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:55.044171 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:55.046670 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:55.046683 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:55.046683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:55.046696 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:55.048404 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:55.048804 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:55.048808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:55.048808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:55.048898 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:55.049112 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:55.049113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:55.049114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:55.049338 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:55.049518 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:55.049865 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 03:36:55.050019 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
I20260809 03:36:55.051222 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:55.051268 27579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a76612802d294da095fba5d05b247b90: Generated new TSK 0
I20260809 03:36:55.051473 27579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:55.059538 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:55.052885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41611 (local address 127.27.40.190:34615)
0809 03:36:55.052922 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:55.052924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:55.052989 (+ 65us) server_negotiation.cc:407] Connection header received
0809 03:36:55.053080 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:55.053080 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:55.053088 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:55.053133 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:55.053839 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:55.054321 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:55.055361 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:55.055560 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:55.055610 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:55.056988 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:55.056989 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:55.056990 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:55.056995 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:55.058447 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:55.058844 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:55.058845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:55.058846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:55.058895 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:55.059110 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:55.059111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:55.059111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:55.059213 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:55.059281 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:55.059422 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 03:36:55.059480 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:36:55.061682 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 27474
2026-08-09T03:36:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1519 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:55 dist-test-slave-gkw7 krb5kdc[27604](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:55 dist-test-slave-gkw7 krb5kdc[27604](info): set up 2 sockets
Aug 09 03:36:55 dist-test-slave-gkw7 krb5kdc[27604](info): commencing operation
krb5kdc: starting...
W20260809 03:36:56.294124 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:56 dist-test-slave-gkw7 krb5kdc[27604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:36:56.418529 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:35161
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:35161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:56.501755 27624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:56.501981 27624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:56.502004 27624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:56.503211 27624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:56.503260 27624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:56.503275 27624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:56.503306 27624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:56.503333 27624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:56.504815 27624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:35161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:35161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:56.505056 27624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:56.505506 27624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:56.510931 27633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:56.511149 27624 server_base.cc:1034] running on GCE node
W20260809 03:36:56.511077 27630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:56.510954 27631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:56.511546 27624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:56.511845 27624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:56.513021 27624 hybrid_clock.cc:648] HybridClock initialized: now 1786246616513016 us; error 39 us; skew 500 ppm
Aug 09 03:36:56 dist-test-slave-gkw7 krb5kdc[27604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246616, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:56.519512 27624 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:36:56.520017 27624 webserver.cc:460] Webserver started at http://127.27.40.190:34105/ using document root <none> and password file <none>
I20260809 03:36:56.520262 27624 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:56.520331 27624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:56.520488 27624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:56.521677 27624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "8535e06bb9eb4a8b9b9480ec8fcacf1d"
format_stamp: "Formatted at 2026-08-09 03:36:56 on dist-test-slave-gkw7"
server_key: "9584c320ec336c43a47010c15c06aa72"
server_key_iv: "620a59f1250a5ca5430efb9b36321191"
server_key_version: "encryptionkey@0"
I20260809 03:36:56.522068 27624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "8535e06bb9eb4a8b9b9480ec8fcacf1d"
format_stamp: "Formatted at 2026-08-09 03:36:56 on dist-test-slave-gkw7"
server_key: "9584c320ec336c43a47010c15c06aa72"
server_key_iv: "620a59f1250a5ca5430efb9b36321191"
server_key_version: "encryptionkey@0"
I20260809 03:36:56.523478 27624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 03:36:56.524288 27644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:56.524493 27624 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:56.524609 27624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "8535e06bb9eb4a8b9b9480ec8fcacf1d"
format_stamp: "Formatted at 2026-08-09 03:36:56 on dist-test-slave-gkw7"
server_key: "9584c320ec336c43a47010c15c06aa72"
server_key_iv: "620a59f1250a5ca5430efb9b36321191"
server_key_version: "encryptionkey@0"
I20260809 03:36:56.524703 27624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:56.556084 27624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:56.558684 27624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:56.558862 27624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:56.563336 27624 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:35161
I20260809 03:36:56.563380 27720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:35161 every 8 connection(s)
I20260809 03:36:56.563778 27624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:56.564227 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 27624
I20260809 03:36:56.564350 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:56.564553 27810 external_mini_cluster.cc:1428] Setting key bfaee90ac61946698e5a3aeb762c8058
I20260809 03:36:56.564990 27721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:56.568058 27721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d: Bootstrap starting.
I20260809 03:36:56.568658 27721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:56.568989 27721 log.cc:824] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:56.569803 27721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d: No bootstrap required, opened a new log
I20260809 03:36:56.570402 27724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:56.565114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42999 (local address 127.27.40.190:35161)
0809 03:36:56.565320 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:56.565331 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:56.565429 (+ 98us) server_negotiation.cc:407] Connection header received
0809 03:36:56.566261 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:56.566269 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:56.566321 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:56.566503 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:56.566743 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:56.567622 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:56.568488 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:56.568935 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:56.569053 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:56.569741 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:56.569751 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:56.569751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:56.569937 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:56.569973 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:56.569993 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:56.570064 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:56.570131 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 03:36:56.570214 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":112,"threads_started":1}
I20260809 03:36:56.571441 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:56.572310 27721 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535e06bb9eb4a8b9b9480ec8fcacf1d" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35161 } }
I20260809 03:36:56.572504 27721 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:56.572561 27721 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8535e06bb9eb4a8b9b9480ec8fcacf1d, State: Initialized, Role: FOLLOWER
I20260809 03:36:56.572702 27721 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535e06bb9eb4a8b9b9480ec8fcacf1d" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35161 } }
I20260809 03:36:56.572863 27721 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:56.572921 27721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:56.572984 27721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:56.573642 27721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535e06bb9eb4a8b9b9480ec8fcacf1d" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35161 } }
I20260809 03:36:56.573774 27721 leader_election.cc:302] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8535e06bb9eb4a8b9b9480ec8fcacf1d; no voters:
I20260809 03:36:56.574008 27721 leader_election.cc:288] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:56.574038 27738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:56.574183 27738 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [term 1 LEADER]: Becoming Leader. State: Replica: 8535e06bb9eb4a8b9b9480ec8fcacf1d, State: Running, Role: LEADER
I20260809 03:36:56.574311 27721 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:56.574338 27738 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535e06bb9eb4a8b9b9480ec8fcacf1d" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35161 } }
I20260809 03:36:56.574746 27738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8535e06bb9eb4a8b9b9480ec8fcacf1d. Latest consensus state: current_term: 1 leader_uuid: "8535e06bb9eb4a8b9b9480ec8fcacf1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535e06bb9eb4a8b9b9480ec8fcacf1d" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35161 } } }
I20260809 03:36:56.574755 27739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8535e06bb9eb4a8b9b9480ec8fcacf1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535e06bb9eb4a8b9b9480ec8fcacf1d" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 35161 } } }
I20260809 03:36:56.574927 27738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:56.574939 27739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:56.575289 27754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:56.576135 27724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:56.572812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45345 (local address 127.27.40.190:35161)
0809 03:36:56.572847 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:56.572849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:56.572955 (+ 106us) server_negotiation.cc:407] Connection header received
0809 03:36:56.573036 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:56.573037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:56.573047 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:56.573103 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:56.573197 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:56.573727 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:56.574684 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:56.574963 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:56.575026 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:56.575737 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:56.575738 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:56.575739 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:56.575743 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:56.575755 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:56.575759 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:56.575801 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:56.576038 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 03:36:56.576063 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:36:56.576407 27754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:56.577992 27754 catalog_manager.cc:1347] Generated new cluster ID: 8e494c03e961433b8642ab0a6eefeb45
I20260809 03:36:56.578053 27754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:56.611181 27754 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:56.611774 27754 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:56.617667 27754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8535e06bb9eb4a8b9b9480ec8fcacf1d: Generated new TSK 0
I20260809 03:36:56.617872 27754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:56.625891 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 27624
2026-08-09T03:36:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1559 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:56 dist-test-slave-gkw7 krb5kdc[27769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:56 dist-test-slave-gkw7 krb5kdc[27769](info): set up 2 sockets
Aug 09 03:36:56 dist-test-slave-gkw7 krb5kdc[27769](info): commencing operation
krb5kdc: starting...
W20260809 03:36:57.847635 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:57 dist-test-slave-gkw7 krb5kdc[27769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:36:57.969026 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:38959
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:38959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:58.050827 27786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:58.051040 27786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:58.051062 27786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:58.052142 27786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:58.052178 27786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:58.052192 27786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:58.052205 27786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:58.052217 27786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:58.053694 27786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:38959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:38959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:58.053879 27786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:58.054040 27786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:58.058810 27794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:58.058835 27796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:58.058876 27793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:58.059252 27786 server_base.cc:1034] running on GCE node
I20260809 03:36:58.059500 27786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:58.059788 27786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:58.060969 27786 hybrid_clock.cc:648] HybridClock initialized: now 1786246618060960 us; error 43 us; skew 500 ppm
Aug 09 03:36:58 dist-test-slave-gkw7 krb5kdc[27769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246618, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:58.063661 27786 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:36:58.064142 27786 webserver.cc:460] Webserver started at http://127.27.40.190:34595/ using document root <none> and password file <none>
I20260809 03:36:58.064369 27786 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:58.064436 27786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:58.064592 27786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:58.065742 27786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "c62eb9691c2d4076986ba912699474ef"
format_stamp: "Formatted at 2026-08-09 03:36:58 on dist-test-slave-gkw7"
server_key: "e7877570ca9600bda1bde85bc49a7acd"
server_key_iv: "2564a05f3fe5e78566efeb3b5d5304fe"
server_key_version: "encryptionkey@0"
I20260809 03:36:58.066121 27786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "c62eb9691c2d4076986ba912699474ef"
format_stamp: "Formatted at 2026-08-09 03:36:58 on dist-test-slave-gkw7"
server_key: "e7877570ca9600bda1bde85bc49a7acd"
server_key_iv: "2564a05f3fe5e78566efeb3b5d5304fe"
server_key_version: "encryptionkey@0"
I20260809 03:36:58.067461 27786 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:58.068209 27808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:58.068375 27786 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:36:58.068481 27786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "c62eb9691c2d4076986ba912699474ef"
format_stamp: "Formatted at 2026-08-09 03:36:58 on dist-test-slave-gkw7"
server_key: "e7877570ca9600bda1bde85bc49a7acd"
server_key_iv: "2564a05f3fe5e78566efeb3b5d5304fe"
server_key_version: "encryptionkey@0"
I20260809 03:36:58.068557 27786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:58.071907 27786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:58.074229 27786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:58.074391 27786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:58.078636 27786 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:38959
I20260809 03:36:58.078670 27894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:38959 every 8 connection(s)
I20260809 03:36:58.078995 27786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:58.079999 27895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:58.082407 27895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef: Bootstrap starting.
I20260809 03:36:58.082988 27895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:58.083297 27895 log.cc:824] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:58.083454 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 27786
I20260809 03:36:58.083585 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:58.083820 27810 external_mini_cluster.cc:1428] Setting key cdad5f5ae0bc2a978b97c271eeb050e7
I20260809 03:36:58.084111 27895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef: No bootstrap required, opened a new log
I20260809 03:36:58.085922 27895 raft_consensus.cc:348] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62eb9691c2d4076986ba912699474ef" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38959 } }
I20260809 03:36:58.086071 27895 raft_consensus.cc:374] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:58.086138 27895 raft_consensus.cc:729] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62eb9691c2d4076986ba912699474ef, State: Initialized, Role: FOLLOWER
I20260809 03:36:58.086318 27895 consensus_queue.cc:260] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62eb9691c2d4076986ba912699474ef" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38959 } }
I20260809 03:36:58.086433 27895 raft_consensus.cc:388] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:58.086468 27895 raft_consensus.cc:482] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:58.086560 27895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:58.087278 27902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:58.084350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56787 (local address 127.27.40.190:38959)
0809 03:36:58.084563 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:58.084572 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:58.084618 (+ 46us) server_negotiation.cc:407] Connection header received
0809 03:36:58.085907 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:58.085914 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:58.085942 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:58.086088 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:58.086829 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:58.086833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:58.086833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:58.086837 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:58.086886 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:58.086896 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:58.086966 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:58.087043 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 03:36:58.087085 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260809 03:36:58.087638 27895 raft_consensus.cc:504] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62eb9691c2d4076986ba912699474ef" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38959 } }
I20260809 03:36:58.087745 27895 leader_election.cc:302] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c62eb9691c2d4076986ba912699474ef; no voters:
I20260809 03:36:58.087939 27895 leader_election.cc:288] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:58.087962 27903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:58.088178 27903 raft_consensus.cc:686] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [term 1 LEADER]: Becoming Leader. State: Replica: c62eb9691c2d4076986ba912699474ef, State: Running, Role: LEADER
I20260809 03:36:58.088241 27895 sys_catalog.cc:564] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:58.088274 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:58.088296 27903 consensus_queue.cc:237] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62eb9691c2d4076986ba912699474ef" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38959 } }
I20260809 03:36:58.088609 27904 sys_catalog.cc:455] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c62eb9691c2d4076986ba912699474ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62eb9691c2d4076986ba912699474ef" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38959 } } }
I20260809 03:36:58.088680 27904 sys_catalog.cc:458] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:58.088878 27905 sys_catalog.cc:455] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader c62eb9691c2d4076986ba912699474ef. Latest consensus state: current_term: 1 leader_uuid: "c62eb9691c2d4076986ba912699474ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62eb9691c2d4076986ba912699474ef" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 38959 } } }
I20260809 03:36:58.088927 27905 sys_catalog.cc:458] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:58.089023 27910 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:58.089574 27910 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:58.091058 27902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:58.090099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34623 (local address 127.27.40.190:38959)
0809 03:36:58.090132 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:58.090135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:58.090144 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:36:58.090163 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:58.090164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:58.090168 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:58.090223 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:58.090813 (+ 590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:58.090815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:58.090815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:36:58.090819 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:58.090833 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:36:58.090837 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:58.090866 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:58.090983 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 03:36:58.090998 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:36:58.093954 27910 catalog_manager.cc:1347] Generated new cluster ID: 766150386929432b818d53b799fabd41
I20260809 03:36:58.094015 27910 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:58.131326 27910 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:58.131539 27910 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:58.138298 27910 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c62eb9691c2d4076986ba912699474ef: Generated new TSK 0
I20260809 03:36:58.138507 27910 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:36:58.144232 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 27786
2026-08-09T03:36:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1519 ms)
[----------] 18 tests from AuthTokenIssuingTest (27679 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:58 dist-test-slave-gkw7 krb5kdc[27937](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:58 dist-test-slave-gkw7 krb5kdc[27937](info): set up 2 sockets
Aug 09 03:36:58 dist-test-slave-gkw7 krb5kdc[27937](info): commencing operation
krb5kdc: starting...
W20260809 03:36:59.391726 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:36:59 dist-test-slave-gkw7 krb5kdc[27937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:36:59.514598 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:41487
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:41487,127.27.40.189:35331,127.27.40.188:35647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:59.596410 27955 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:36:59.596637 27955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:59.596657 27955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:59.596678 27955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:59.598022 27955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:59.598069 27955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:59.598084 27955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:59.598097 27955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:59.598110 27955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:59.599699 27955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:41487,127.27.40.189:35331,127.27.40.188:35647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:41487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:59.599977 27955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:59.600219 27955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:59.605206 27964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:59.605193 27962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:59.605194 27969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:59.605381 27955 server_base.cc:1034] running on GCE node
I20260809 03:36:59.605772 27955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:59.606012 27955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:59.607188 27955 hybrid_clock.cc:648] HybridClock initialized: now 1786246619607167 us; error 41 us; skew 500 ppm
Aug 09 03:36:59 dist-test-slave-gkw7 krb5kdc[27937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246619, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:59.609738 27955 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:36:59.610185 27955 webserver.cc:460] Webserver started at http://127.27.40.190:34435/ using document root <none> and password file <none>
I20260809 03:36:59.610455 27955 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:59.610526 27955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:59.610698 27955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:59.611737 27955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "ad06b9de24c2461f8490a89f37ce6185"
format_stamp: "Formatted at 2026-08-09 03:36:59 on dist-test-slave-gkw7"
server_key: "cc1d8808001b115877b22aeaf93e8727"
server_key_iv: "8addeba002c1c7d13e208cb4c66bb64d"
server_key_version: "encryptionkey@0"
I20260809 03:36:59.612098 27955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "ad06b9de24c2461f8490a89f37ce6185"
format_stamp: "Formatted at 2026-08-09 03:36:59 on dist-test-slave-gkw7"
server_key: "cc1d8808001b115877b22aeaf93e8727"
server_key_iv: "8addeba002c1c7d13e208cb4c66bb64d"
server_key_version: "encryptionkey@0"
I20260809 03:36:59.613481 27955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 03:36:59.614449 27978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:59.614673 27955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:36:59.614781 27955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "ad06b9de24c2461f8490a89f37ce6185"
format_stamp: "Formatted at 2026-08-09 03:36:59 on dist-test-slave-gkw7"
server_key: "cc1d8808001b115877b22aeaf93e8727"
server_key_iv: "8addeba002c1c7d13e208cb4c66bb64d"
server_key_version: "encryptionkey@0"
I20260809 03:36:59.614866 27955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:59.626673 27955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:59.628539 27955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:59.628706 27955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:59.632911 27955 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:41487
I20260809 03:36:59.632952 28052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:41487 every 8 connection(s)
I20260809 03:36:59.633333 27955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:59.634395 28053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:59.635957 28053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:36:59.639719 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 27955
I20260809 03:36:59.639848 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:36:59.640030 27810 external_mini_cluster.cc:1428] Setting key e637a2222a313b725d9800c0d314ad0d
Aug 09 03:36:59 dist-test-slave-gkw7 krb5kdc[27937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246619, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:59.647051 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.636787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54982 (local address 127.27.40.190:41487)
0809 03:36:59.636902 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:59.636907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:59.636939 (+ 32us) server_negotiation.cc:407] Connection header received
0809 03:36:59.637497 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:59.637499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:59.637570 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:59.637700 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:59.638235 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.639096 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.640020 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.640380 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.640520 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.643232 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:59.643240 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:59.643240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:59.643247 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:59.645320 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.645822 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.645825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.645826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.645933 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.646228 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.646229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.646229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.646430 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:59.646527 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:59.646799 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 03:36:59.646899 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":39,"threads_started":1}
I20260809 03:36:59.647253 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.636626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:41487 (local address 127.0.0.1:54982)
0809 03:36:59.636782 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.636816 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:59.637334 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:59.637689 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:59.637692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:59.637965 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:59.638133 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:59.638138 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.639194 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.639195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:59.639929 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:59.639932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.640496 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.640497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:59.640562 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.641130 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:59.641138 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:59.643128 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:59.645425 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:59.645436 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:59.645438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:59.645733 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:59.646008 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:59.646009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:59.646009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:59.646155 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:59.646543 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:59.646552 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:59.646643 (+ 91us) client_negotiation.cc:769] Sending connection context
0809 03:36:59.646764 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 03:36:59.647014 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":56,"thread_start_us":73,"threads_started":1}
I20260809 03:36:59.648012 28053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:36:59.648387 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.648246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:35331 (local address 127.0.0.1:43198)
0809 03:36:59.648300 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.648361 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:35331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 03:36:59.648564 27980 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.40.189:35331: connect: Connection refused (error 111)
I20260809 03:36:59.649063 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.648967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:35331 (local address 127.0.0.1:43210)
0809 03:36:59.649018 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.649045 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:35331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 03:36:59.649197 28053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.189:35331: Network error: Client connection negotiation failed: client connection to 127.27.40.189:35331: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.27.40.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.189.keytab.
Entry for principal kudu/127.27.40.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.189.keytab.
I20260809 03:36:59.667765 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.189:35331
--webserver_interface=127.27.40.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.189.keytab
--principal=kudu/127.27.40.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:41487,127.27.40.189:35331,127.27.40.188:35647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:36:59.698426 28053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } attempt: 1
I20260809 03:36:59.698904 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.698766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:35331 (local address 127.0.0.1:43218)
0809 03:36:59.698835 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.698874 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:35331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 03:36:59.699276 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.699178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:35331 (local address 127.0.0.1:43222)
0809 03:36:59.699225 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.699252 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:35331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 03:36:59.699365 28053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.189:35331: Network error: Client connection negotiation failed: client connection to 127.27.40.189:35331: connect: Connection refused (error 111)
W20260809 03:36:59.749382 28066 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:36:59.749658 28066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:59.749691 28066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:59.749730 28066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:59.750855 28066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:59.750919 28066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:59.750946 28066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:59.750975 28066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:59.750998 28066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:59.752513 28066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:41487,127.27.40.189:35331,127.27.40.188:35647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.189.keytab
--principal=kudu/127.27.40.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.189:35331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.40.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:59.752799 28066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:59.753021 28066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:59.757879 28081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:59.757879 28086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:59.757896 28082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:59.758051 28066 server_base.cc:1034] running on GCE node
I20260809 03:36:59.758358 28066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:59.758625 28066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:59.759763 28066 hybrid_clock.cc:648] HybridClock initialized: now 1786246619759752 us; error 28 us; skew 500 ppm
Aug 09 03:36:59 dist-test-slave-gkw7 krb5kdc[27937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246619, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:59.762279 28066 init.cc:377] Logged in from keytab as kudu/127.27.40.189@KRBTEST.COM (short username kudu)
I20260809 03:36:59.762727 28066 webserver.cc:460] Webserver started at http://127.27.40.189:33031/ using document root <none> and password file <none>
I20260809 03:36:59.762960 28066 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:59.763029 28066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:59.763186 28066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:59.764230 28066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/data/instance:
uuid: "96ba2f5ea2f6495c8fe5817699e301d5"
format_stamp: "Formatted at 2026-08-09 03:36:59 on dist-test-slave-gkw7"
server_key: "6e7a39ea2dbc54e684145728e8d8c245"
server_key_iv: "693490e5d5b050070d35000795c3cfbb"
server_key_version: "encryptionkey@0"
I20260809 03:36:59.764598 28066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/wal/instance:
uuid: "96ba2f5ea2f6495c8fe5817699e301d5"
format_stamp: "Formatted at 2026-08-09 03:36:59 on dist-test-slave-gkw7"
server_key: "6e7a39ea2dbc54e684145728e8d8c245"
server_key_iv: "693490e5d5b050070d35000795c3cfbb"
server_key_version: "encryptionkey@0"
I20260809 03:36:59.766014 28066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:36:59.766742 28094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:59.766913 28066 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 03:36:59.767030 28066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/wal
uuid: "96ba2f5ea2f6495c8fe5817699e301d5"
format_stamp: "Formatted at 2026-08-09 03:36:59 on dist-test-slave-gkw7"
server_key: "6e7a39ea2dbc54e684145728e8d8c245"
server_key_iv: "693490e5d5b050070d35000795c3cfbb"
server_key_version: "encryptionkey@0"
I20260809 03:36:59.767105 28066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:59.767508 28053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } attempt: 2
I20260809 03:36:59.767899 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.767768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:35331 (local address 127.0.0.1:43238)
0809 03:36:59.767830 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.767866 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:35331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 03:36:59.768271 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.768204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:35331 (local address 127.0.0.1:43246)
0809 03:36:59.768226 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.768249 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:35331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260809 03:36:59.768379 28053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.189:35331: Network error: Client connection negotiation failed: client connection to 127.27.40.189:35331: connect: Connection refused (error 111)
I20260809 03:36:59.777375 28066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:59.779325 28066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:59.779465 28066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:59.783550 28066 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.189:35331
I20260809 03:36:59.783603 28174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.189:35331 every 8 connection(s)
I20260809 03:36:59.783957 28066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/data/info.pb
I20260809 03:36:59.784924 28175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:59.786509 28175 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:36:59 dist-test-slave-gkw7 krb5kdc[27937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246619, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.189@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:59.792635 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 28066
I20260809 03:36:59.792744 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-1/wal/instance
I20260809 03:36:59.792970 27810 external_mini_cluster.cc:1428] Setting key 445013c007967eccae3e7d02c2f2e86f
I20260809 03:36:59.795451 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.787181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54990 (local address 127.27.40.190:41487)
0809 03:36:59.787256 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:59.787258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:59.787449 (+ 191us) server_negotiation.cc:407] Connection header received
0809 03:36:59.787908 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:59.787909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:59.787919 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:59.787974 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:59.788536 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.788994 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.790020 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.790227 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.790282 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.792342 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:59.792343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:59.792343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:59.792350 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:59.794103 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.794490 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.794491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.794491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.794558 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.794815 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.794817 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.794817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.794940 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:59.795012 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:59.795280 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 03:36:59.795364 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:36:59.795637 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.787117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:41487 (local address 127.0.0.1:54990)
0809 03:36:59.787327 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.787370 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:59.787818 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:59.788043 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:59.788046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:59.788285 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:59.788464 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:59.788467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.789070 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.789071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:59.789933 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:59.789935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.790331 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.790332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:59.790434 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.790930 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:59.790938 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:59.792253 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:59.794203 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:59.794210 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:59.794211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:59.794418 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:59.794628 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:59.794629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:59.794629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:59.794739 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:59.795016 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:59.795023 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:59.795151 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 03:36:59.795259 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 03:36:59.795402 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":97,"threads_started":1}
I20260809 03:36:59.796200 28175 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:36:59 dist-test-slave-gkw7 krb5kdc[27937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246619, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.189@KRBTEST.COM for kudu/127.27.40.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.40.188@KRBTEST.COM; defaulting to no policy
I20260809 03:36:59.803987 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.796433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:35331 (local address 127.0.0.1:43248)
0809 03:36:59.796479 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.796490 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:59.796527 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:59.796808 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:59.796809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:59.796937 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:59.796980 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:59.796981 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.798139 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.798140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:59.798640 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:59.798641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.799146 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.799147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:59.799210 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.799649 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:59.799654 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:59.801013 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:59.802919 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:59.802920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:59.802921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:59.803085 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:59.803315 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:59.803316 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:59.803316 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:59.803371 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:59.803678 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:59.803679 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:59.803718 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 03:36:59.803783 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 03:36:59.803881 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 03:36:59.804057 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.796534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43248 (local address 127.27.40.189:35331)
0809 03:36:59.796666 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:59.796670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:59.796686 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:36:59.796711 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:59.796712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:59.796722 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:59.796795 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:59.797060 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.798034 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.798730 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.799054 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.799147 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.801100 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:59.801108 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:59.801109 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:59.801115 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:59.802835 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.803185 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.803187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.803188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.803243 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.803442 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.803443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.803443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.803585 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:59.803687 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:59.803820 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 03:36:59.803932 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.188@KRBTEST.COM" created.
I20260809 03:36:59.804571 28175 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:36:59.804927 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.804801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:35647 (local address 127.0.0.1:49668)
0809 03:36:59.804856 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.804907 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 03:36:59.805104 28095 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.40.188:35647: connect: Connection refused (error 111)
I20260809 03:36:59.805516 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.805434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:35647 (local address 127.0.0.1:49680)
0809 03:36:59.805475 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.805498 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 03:36:59.805671 28175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.188:35647: Network error: Client connection negotiation failed: client connection to 127.27.40.188:35647: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.188.keytab.
Entry for principal kudu/127.27.40.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.188.keytab.
I20260809 03:36:59.818771 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.188:35647
--webserver_interface=127.27.40.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.188.keytab
--principal=kudu/127.27.40.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:41487,127.27.40.189:35331,127.27.40.188:35647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:36:59.854899 28175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } attempt: 1
I20260809 03:36:59.855455 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.855266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:35647 (local address 127.0.0.1:49696)
0809 03:36:59.855350 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.855407 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 03:36:59.855854 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.855747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:35647 (local address 127.0.0.1:49712)
0809 03:36:59.855798 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.855828 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 03:36:59.855997 28175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.188:35647: Network error: Client connection negotiation failed: client connection to 127.27.40.188:35647: connect: Connection refused (error 111)
I20260809 03:36:59.859546 28053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } attempt: 3
Aug 09 03:36:59 dist-test-slave-gkw7 krb5kdc[27937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246619, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for kudu/127.27.40.189@KRBTEST.COM
I20260809 03:36:59.866971 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.859892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:35331 (local address 127.0.0.1:43254)
0809 03:36:59.859954 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.859965 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:59.860028 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:59.860171 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:59.860171 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:59.860359 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:59.860437 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:59.860439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.861086 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.861087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:59.861981 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:59.861984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.862406 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.862407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:59.862488 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.862898 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:59.862903 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:59.864285 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:59.866035 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:59.866036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:59.866036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:59.866250 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:59.866456 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:59.866457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:59.866457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:59.866509 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:59.866753 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:59.866754 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:59.866790 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 03:36:59.866839 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 03:36:59.866910 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 03:36:59.867036 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.859981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43254 (local address 127.27.40.189:35331)
0809 03:36:59.860039 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:59.860043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:59.860057 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:36:59.860080 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:59.860081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:59.860090 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:59.860160 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:59.860513 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.861002 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.862065 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.862291 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.862387 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.864369 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:59.864371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:59.864371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:59.864379 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:59.865944 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.866330 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.866331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.866332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.866393 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.866565 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.866566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.866566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.866695 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:59.866760 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:59.866881 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 03:36:59.866958 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 03:36:59.867688 28053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:36:59.868005 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.867910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:35647 (local address 127.0.0.1:49726)
0809 03:36:59.867954 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.867983 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 03:36:59.868302 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.868249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:35647 (local address 127.0.0.1:49736)
0809 03:36:59.868269 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.868289 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260809 03:36:59.868407 28053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.188:35647: Network error: Client connection negotiation failed: client connection to 127.27.40.188:35647: connect: Connection refused (error 111)
I20260809 03:36:59.899586 28053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } attempt: 1
I20260809 03:36:59.900071 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.899915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:35647 (local address 127.0.0.1:49744)
0809 03:36:59.899989 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.900035 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 03:36:59.900473 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.900363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:35647 (local address 127.0.0.1:49760)
0809 03:36:59.900417 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.900448 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 03:36:59.900579 28053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.188:35647: Network error: Client connection negotiation failed: client connection to 127.27.40.188:35647: connect: Connection refused (error 111)
W20260809 03:36:59.903477 28188 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:36:59.903669 28188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:59.903724 28188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:59.903766 28188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:59.904904 28188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:59.904943 28188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:59.904958 28188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:59.905009 28188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:59.905028 28188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:59.906581 28188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:41487,127.27.40.189:35331,127.27.40.188:35647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.188.keytab
--principal=kudu/127.27.40.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.188:35647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.40.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:36:59.906841 28188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:59.907063 28188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:59.911823 28199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:59.911873 28195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:59.911855 28196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:59.912034 28188 server_base.cc:1034] running on GCE node
I20260809 03:36:59.912350 28188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:59.912654 28188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:59.913789 28188 hybrid_clock.cc:648] HybridClock initialized: now 1786246619913769 us; error 35 us; skew 500 ppm
Aug 09 03:36:59 dist-test-slave-gkw7 krb5kdc[27937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246619, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:59.916160 28188 init.cc:377] Logged in from keytab as kudu/127.27.40.188@KRBTEST.COM (short username kudu)
I20260809 03:36:59.916529 28188 webserver.cc:460] Webserver started at http://127.27.40.188:34949/ using document root <none> and password file <none>
I20260809 03:36:59.916729 28188 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:59.916788 28188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:59.916877 28188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:59.917974 28188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/data/instance:
uuid: "f6228cf27d864568ba24c87466ffb5a4"
format_stamp: "Formatted at 2026-08-09 03:36:59 on dist-test-slave-gkw7"
server_key: "42d113611a266d3bdb1b08ced5c52b5e"
server_key_iv: "9a1d58407a75e47943fcc4f902f7e08f"
server_key_version: "encryptionkey@0"
I20260809 03:36:59.918282 28188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/wal/instance:
uuid: "f6228cf27d864568ba24c87466ffb5a4"
format_stamp: "Formatted at 2026-08-09 03:36:59 on dist-test-slave-gkw7"
server_key: "42d113611a266d3bdb1b08ced5c52b5e"
server_key_iv: "9a1d58407a75e47943fcc4f902f7e08f"
server_key_version: "encryptionkey@0"
I20260809 03:36:59.919535 28188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:36:59.920228 28208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:59.920382 28188 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 03:36:59.920468 28188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/wal
uuid: "f6228cf27d864568ba24c87466ffb5a4"
format_stamp: "Formatted at 2026-08-09 03:36:59 on dist-test-slave-gkw7"
server_key: "42d113611a266d3bdb1b08ced5c52b5e"
server_key_iv: "9a1d58407a75e47943fcc4f902f7e08f"
server_key_version: "encryptionkey@0"
I20260809 03:36:59.920521 28188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:59.924166 28175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } attempt: 2
I20260809 03:36:59.924536 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.924408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:35647 (local address 127.0.0.1:49776)
0809 03:36:59.924476 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.924510 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 03:36:59.924855 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.924757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:35647 (local address 127.0.0.1:49792)
0809 03:36:59.924802 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.924826 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 03:36:59.924966 28175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.188:35647: Network error: Client connection negotiation failed: client connection to 127.27.40.188:35647: connect: Connection refused (error 111)
I20260809 03:36:59.943849 28188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:59.946113 28188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:59.946318 28188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:59.950472 28188 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.188:35647
I20260809 03:36:59.950513 28279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.188:35647 every 8 connection(s)
I20260809 03:36:59.950896 28188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/data/info.pb
I20260809 03:36:59.952105 28282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:59.953716 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 28188
I20260809 03:36:59.953850 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246339702443-27810-0/minicluster-data/master-2/wal/instance
I20260809 03:36:59.954097 27810 external_mini_cluster.cc:1428] Setting key 68fb394b300c4711f13122e4ffef0174
I20260809 03:36:59.954228 28282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:36:59 dist-test-slave-gkw7 krb5kdc[27937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246619, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.188@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
Aug 09 03:36:59 dist-test-slave-gkw7 krb5kdc[27937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.188@KRBTEST.COM
I20260809 03:36:59.964576 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.955071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55006 (local address 127.27.40.190:41487)
0809 03:36:59.955134 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:59.955136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:59.955298 (+ 162us) server_negotiation.cc:407] Connection header received
0809 03:36:59.955798 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:59.955799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:59.955811 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:59.955878 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:59.956663 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.957126 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.958316 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.958625 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.958684 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.961185 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:59.961188 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:59.961188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:59.961196 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:59.962904 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.963438 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.963439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.963439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.963492 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.963790 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.963791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.963792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.963931 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:59.964018 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:59.964392 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 03:36:59.964482 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 03:36:59.964756 28286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.954975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:41487 (local address 127.0.0.1:55006)
0809 03:36:59.955150 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.955194 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:59.955667 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:59.955922 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:59.955925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:59.956294 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:59.956545 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:59.956551 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.957210 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.957211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:59.958231 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:59.958233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.958732 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.958734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:59.958896 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.959493 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:59.959501 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:59.961067 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:59.963005 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:59.963010 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:59.963012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:59.963366 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:59.963587 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:59.963588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:59.963588 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:59.963721 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:59.964021 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:59.964026 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:59.964215 (+ 189us) client_negotiation.cc:769] Sending connection context
0809 03:36:59.964332 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 03:36:59.964484 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
I20260809 03:36:59.964758 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.954693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49796 (local address 127.27.40.188:35647)
0809 03:36:59.954907 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:59.954915 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:59.954973 (+ 58us) server_negotiation.cc:407] Connection header received
0809 03:36:59.955663 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:59.955673 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:59.955703 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:59.955808 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:59.956470 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.957651 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.958571 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.959013 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.959103 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.961764 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:59.961771 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:59.961771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:59.961787 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:59.963525 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.963889 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.963891 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.963892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.963938 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.964151 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.964152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.964152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.964308 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:59.964436 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:59.964551 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 03:36:59.964618 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":110,"threads_started":1}
I20260809 03:36:59.965719 28282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:36:59 dist-test-slave-gkw7 krb5kdc[27937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246619, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.188@KRBTEST.COM for kudu/127.27.40.189@KRBTEST.COM
Aug 09 03:36:59 dist-test-slave-gkw7 krb5kdc[27937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:36:59.972714 28286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.965966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:35331 (local address 127.0.0.1:43264)
0809 03:36:59.966009 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.966019 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:59.966053 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:59.966200 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:59.966201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:59.966368 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:59.966418 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:59.966419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.967061 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.967061 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:59.967658 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:59.967660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.967992 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.967993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:59.968058 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.968449 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:59.968454 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:59.969917 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:59.971676 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:59.971677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:59.971678 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:59.971842 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:59.972049 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:59.972050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:59.972051 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:59.972115 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:59.972414 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:59.972415 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:59.972455 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 03:36:59.972539 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 03:36:59.972625 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 03:36:59.972734 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.966043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43264 (local address 127.27.40.189:35331)
0809 03:36:59.966091 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:59.966093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:59.966101 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:36:59.966125 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:59.966126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:59.966135 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:59.966188 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:59.966483 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.966983 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.967736 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.967926 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.967975 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.969994 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:59.969995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:59.969995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:59.970001 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:59.971574 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.971916 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.971917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.971917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.971982 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.972187 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.972188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.972188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.972347 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:59.972421 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:59.972565 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 03:36:59.972650 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 03:36:59.973280 28282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:36:59.973738 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.966032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55008 (local address 127.27.40.190:41487)
0809 03:36:59.966068 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:59.966069 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:59.966298 (+ 229us) server_negotiation.cc:407] Connection header received
0809 03:36:59.966350 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:59.966351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:59.966358 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:59.966404 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:59.966894 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.967403 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.968252 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.968426 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.968505 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.970876 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:59.970877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:59.970877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:59.970885 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:59.972382 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.972735 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.972736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.972736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.972792 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.972990 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.972991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.972992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.973126 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:59.973285 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:59.973338 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 03:36:59.973619 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 03:36:59.975737 28053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } attempt: 2
Aug 09 03:36:59 dist-test-slave-gkw7 krb5kdc[27937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246619, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.188@KRBTEST.COM for kudu/127.27.40.188@KRBTEST.COM
Aug 09 03:36:59 dist-test-slave-gkw7 krb5kdc[27937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.189@KRBTEST.COM
I20260809 03:36:59.980525 28286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.973553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:35647 (local address 127.0.0.1:49812)
0809 03:36:59.973597 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.973608 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:59.973648 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:59.973786 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:59.973786 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:59.973930 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:59.973959 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:59.973961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.974756 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.974757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:59.975462 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:59.975464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.975889 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.975890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:59.975931 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.976326 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:59.976331 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:59.977620 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:59.979541 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:59.979542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:59.979542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:59.979711 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:59.979915 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:59.979916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:59.979917 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:59.979971 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:59.980217 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:59.980217 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:59.980254 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 03:36:59.980322 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 03:36:59.980417 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 03:36:59.980574 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.973613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49812 (local address 127.27.40.188:35647)
0809 03:36:59.973650 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:59.973652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:59.973660 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:36:59.973694 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:59.973695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:59.973703 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:59.973748 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:59.974022 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.974668 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.975544 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.975789 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.975849 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.977702 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:59.977703 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:59.977703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:59.977710 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:59.979477 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.979789 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.979790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.979790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.979841 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.980043 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.980044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.980044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.980151 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:59.980224 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:59.980346 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 03:36:59.980466 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":18}
Aug 09 03:36:59 dist-test-slave-gkw7 krb5kdc[27937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246619, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for kudu/127.27.40.188@KRBTEST.COM
I20260809 03:36:59.982527 28282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4: Bootstrap starting.
I20260809 03:36:59.982831 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.974348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43276 (local address 127.27.40.189:35331)
0809 03:36:59.974382 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:59.974383 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:59.974390 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:36:59.974412 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:59.974413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:59.974421 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:59.974475 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:59.974985 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.975648 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.976574 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.976873 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.976931 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.979374 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:59.979376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:59.979376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:59.979382 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:59.981311 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.981960 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.981961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.981961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.982022 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.982267 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.982268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.982268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.982397 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:59.982551 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:59.982679 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 03:36:59.982754 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:36:59.983335 28282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:59.983664 28282 log.cc:824] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:59.984560 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:36:59.984761 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.975984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:35647 (local address 127.0.0.1:49826)
0809 03:36:59.976042 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:59.976050 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:59.976085 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:59.976476 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:59.976477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:59.976666 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:59.976700 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:59.976701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.977603 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.977604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:59.978426 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:59.978429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.979457 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.979458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:59.979523 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.979946 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:59.979952 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:59.981386 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:59.983449 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:59.983450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:59.983451 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:59.983701 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:59.984010 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:59.984010 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:59.984011 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:59.984097 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:59.984420 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:59.984421 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:59.984466 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 03:36:59.984554 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 03:36:59.984659 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 03:36:59.984792 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.976154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49826 (local address 127.27.40.188:35647)
0809 03:36:59.976363 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:59.976365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:59.976371 (+ 6us) server_negotiation.cc:407] Connection header received
0809 03:36:59.976385 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:59.976386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:59.976393 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:59.976472 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:59.976780 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.977483 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.978517 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.979369 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.979449 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.981463 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:59.981464 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:59.981464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:59.981471 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:59.983351 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.983788 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.983788 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.983789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.983851 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:59.984175 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:59.984175 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:59.984176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:59.984359 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:59.984427 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:59.984592 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 03:36:59.984663 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":102,"threads_started":1}
I20260809 03:36:59.985900 28282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4: No bootstrap required, opened a new log
I20260809 03:36:59.986443 28053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185: Bootstrap starting.
I20260809 03:36:59.985751 28018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ad06b9de24c2461f8490a89f37ce6185"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55008
I20260809 03:36:59.987125 28053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:59.987424 28053 log.cc:824] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:59.987824 28282 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } }
I20260809 03:36:59.988018 28282 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6228cf27d864568ba24c87466ffb5a4, State: Initialized, Role: FOLLOWER
I20260809 03:36:59.988124 28282 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } }
I20260809 03:36:59.988276 28053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185: No bootstrap required, opened a new log
I20260809 03:36:59.988514 28282 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:59.988507 28291 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } } }
I20260809 03:36:59.988723 28291 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:36:59.989696 28053 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } }
I20260809 03:36:59.989816 28053 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad06b9de24c2461f8490a89f37ce6185, State: Initialized, Role: FOLLOWER
I20260809 03:36:59.989934 28053 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } }
I20260809 03:36:59.990254 28053 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:59.990298 28018 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 03:36:59.990345 28018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:59.990221 28311 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } } }
I20260809 03:36:59.990473 28311 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:36:59.991362 28018 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } }
I20260809 03:36:59.991623 28018 leader_election.cc:288] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [CANDIDATE]: Term 1 election: Requested vote from peers 96ba2f5ea2f6495c8fe5817699e301d5 (127.27.40.189:35331), f6228cf27d864568ba24c87466ffb5a4 (127.27.40.188:35647)
W20260809 03:36:59.991876 28312 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:36:59.991921 28312 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:36:59.992808 28243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad06b9de24c2461f8490a89f37ce6185" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f6228cf27d864568ba24c87466ffb5a4"
I20260809 03:36:59.992938 28243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:59.991305 28188 master_runner.cc:186] Error getting master registration for 127.27.40.190:41487: OK, instance_id { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" instance_seqno: 1786246619631729 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 03:36:59.993882 28243 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad06b9de24c2461f8490a89f37ce6185 in term 1.
I20260809 03:36:59.993991 28188 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 03:36:59.994194 27979 leader_election.cc:302] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad06b9de24c2461f8490a89f37ce6185, f6228cf27d864568ba24c87466ffb5a4; no voters:
I20260809 03:36:59.994359 28311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:59.994576 28311 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [term 1 LEADER]: Becoming Leader. State: Replica: ad06b9de24c2461f8490a89f37ce6185, State: Running, Role: LEADER
I20260809 03:36:59.994685 28311 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } }
I20260809 03:36:59.995188 28311 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad06b9de24c2461f8490a89f37ce6185. Latest consensus state: current_term: 1 leader_uuid: "ad06b9de24c2461f8490a89f37ce6185" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } } }
I20260809 03:36:59.995287 28311 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:59.995541 28347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:59.996650 28347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:59.996785 27955 master_runner.cc:186] Error getting master registration for 127.27.40.189:35331: OK, instance_id { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" instance_seqno: 1786246619782459 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 03:36:59.999387 27955 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 03:37:00.000692 28243 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4 [term 1 FOLLOWER]: Refusing update from remote peer ad06b9de24c2461f8490a89f37ce6185: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:37:00.001008 28311 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:37:00.002533 28291 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad06b9de24c2461f8490a89f37ce6185. Latest consensus state: current_term: 1 leader_uuid: "ad06b9de24c2461f8490a89f37ce6185" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } } }
I20260809 03:37:00.002625 28291 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:37:00.005332 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.992871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49832 (local address 127.27.40.188:35647)
0809 03:36:59.994666 (+ 1795us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:59.994668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:59.994692 (+ 24us) server_negotiation.cc:407] Connection header received
0809 03:36:59.994711 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:59.994712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:59.994722 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:59.994774 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:59.995634 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.996362 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.997362 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:59.997720 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:59.998284 (+ 564us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:59.999601 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:59.999603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:59.999603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:59.999610 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:00.003173 (+ 3563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:00.003791 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:00.003793 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:00.003793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:00.003887 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:00.004201 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:00.004202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:00.004202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:00.004368 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:00.005085 (+ 717us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:00.005133 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 03:37:00.005227 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1753}
I20260809 03:37:00.010282 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.992840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.27.40.189:35331)
0809 03:36:59.992882 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:59.992884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:59.997482 (+ 4598us) server_negotiation.cc:407] Connection header received
0809 03:36:59.997624 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:59.997625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:59.997636 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:59.997669 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:59.999598 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.000318 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.001454 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.002327 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.002374 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:00.006499 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:00.006500 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:00.006501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:00.006509 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:00.008725 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:00.009149 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:00.009150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:00.009151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:00.009222 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:00.009465 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:00.009466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:00.009467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:00.009662 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:00.009783 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:00.010080 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 03:37:00.010176 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 03:37:00.011453 28311 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad06b9de24c2461f8490a89f37ce6185" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } } }
I20260809 03:37:00.011570 28311 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:00.011554 28291 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad06b9de24c2461f8490a89f37ce6185" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } } }
I20260809 03:37:00.011641 28291 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:37:00.011848 28350 mvcc.cc:205] Tried to move back new op lower bound from 7316466155520032768 to 7316466155499057152. Current Snapshot: MvccSnapshot[applied={T|T < 7316466155520032768}]
I20260809 03:37:00.011869 28349 mvcc.cc:205] Tried to move back new op lower bound from 7316466155520032768 to 7316466155499057152. Current Snapshot: MvccSnapshot[applied={T|T < 7316466155520032768}]
I20260809 03:37:00.012907 28347 catalog_manager.cc:1347] Generated new cluster ID: dc69bf1e45df4627ae721ebefb5820b5
I20260809 03:37:00.012962 28347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:37:00.014699 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:59.993283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55020 (local address 127.27.40.190:41487)
0809 03:36:59.993330 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:59.993332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.004885 (+ 11553us) server_negotiation.cc:407] Connection header received
0809 03:37:00.004987 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.004988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.004999 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.005055 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:00.006340 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.007031 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.008514 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.008825 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.008871 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:00.011010 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:00.011011 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:00.011011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:00.011018 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:00.012898 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:00.013513 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:00.013514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:00.013515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:00.013599 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:00.013974 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:00.013975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:00.013975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:00.014104 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:00.014194 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:00.014514 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 03:37:00.014599 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 03:37:00.016093 28175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } attempt: 3
Aug 09 03:37:00 dist-test-slave-gkw7 krb5kdc[27937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246619, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.189@KRBTEST.COM for kudu/127.27.40.188@KRBTEST.COM
I20260809 03:37:00.027896 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:00.016373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:35647 (local address 127.0.0.1:49846)
0809 03:37:00.016419 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:00.016432 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:00.016492 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:00.016713 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:00.016714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:00.016875 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:00.016922 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:00.016924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.018386 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.018388 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:00.019227 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:00.019230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.019963 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.019964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:00.020040 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:00.021169 (+ 1129us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:00.021174 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:00.022703 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:00.026495 (+ 3792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:00.026496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:00.026497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:00.026729 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:00.027063 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:00.027064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:00.027065 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:00.027137 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:00.027591 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:00.027591 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:00.027644 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 03:37:00.027691 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 03:37:00.027786 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 03:37:00.029276 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:00.016396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49846 (local address 127.27.40.188:35647)
0809 03:37:00.016444 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.016446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.016496 (+ 50us) server_negotiation.cc:407] Connection header received
0809 03:37:00.016572 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.016572 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.016583 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.017014 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:00.017074 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.018224 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.019404 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.019857 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.021751 (+ 1894us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:00.022810 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:00.022812 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:00.022812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:00.022819 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:00.026383 (+ 3564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:00.026862 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:00.026863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:00.026863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:00.026945 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:00.027227 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:00.027227 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:00.027228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:00.027476 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:00.028881 (+ 1405us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:00.028952 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 03:37:00.029099 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 03:37:00.031203 28175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5: Bootstrap starting.
I20260809 03:37:00.031981 28175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:00.032265 28175 log.cc:824] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:00.034734 28347 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:37:00.035549 28347 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:37:00.040313 28175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5: No bootstrap required, opened a new log
I20260809 03:37:00.041254 28347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185: Generated new TSK 0
I20260809 03:37:00.041424 28347 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:37:00.042434 28175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } }
I20260809 03:37:00.042567 28175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96ba2f5ea2f6495c8fe5817699e301d5, State: Initialized, Role: FOLLOWER
I20260809 03:37:00.042652 28175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } }
I20260809 03:37:00.043010 28175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:37:00.043224 28135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:00.044126 28135 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5 [term 1 FOLLOWER]: Refusing update from remote peer ad06b9de24c2461f8490a89f37ce6185: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:37:00.044459 28354 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:37:00.043004 28358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } } }
I20260809 03:37:00.044872 28358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:37:00.045014 28136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad06b9de24c2461f8490a89f37ce6185" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "96ba2f5ea2f6495c8fe5817699e301d5"
I20260809 03:37:00.046332 28136 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ad06b9de24c2461f8490a89f37ce6185 for term 1 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260809 03:37:00.047242 28376 catalog_manager.cc:1259] Loaded cluster ID: dc69bf1e45df4627ae721ebefb5820b5
I20260809 03:37:00.047305 28376 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5: loading cluster ID for follower catalog manager: success
I20260809 03:37:00.048097 28376 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5: acquiring CA information for follower catalog manager: success
I20260809 03:37:00.048445 28376 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:37:00.048732 28379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad06b9de24c2461f8490a89f37ce6185. Latest consensus state: current_term: 1 leader_uuid: "ad06b9de24c2461f8490a89f37ce6185" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } } }
I20260809 03:37:00.048796 28379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:37:00.048828 28379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad06b9de24c2461f8490a89f37ce6185" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } } }
I20260809 03:37:00.048887 28379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96ba2f5ea2f6495c8fe5817699e301d5 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 03:37:00.071519 28062 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:00.069685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55026 (local address 127.27.40.190:41487)
0809 03:37:00.069733 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.069735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.070758 (+ 1023us) server_negotiation.cc:407] Connection header received
0809 03:37:00.070891 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.070892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.070904 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.070935 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:00.071449 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55026: BlockingRecv error: recv got EOF from 127.0.0.1:55026 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 03:37:00.072300 28184 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:00.070143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43298 (local address 127.27.40.189:35331)
0809 03:37:00.070182 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.070184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.071731 (+ 1547us) server_negotiation.cc:407] Connection header received
0809 03:37:00.071827 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.071828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.071838 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.071867 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:00.072219 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43298: BlockingRecv error: recv got EOF from 127.0.0.1:43298 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 03:37:00.073007 28287 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:00.070466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49856 (local address 127.27.40.188:35647)
0809 03:37:00.070538 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.070540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.072528 (+ 1988us) server_negotiation.cc:407] Connection header received
0809 03:37:00.072618 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.072619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.072629 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.072659 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:00.072949 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49856: BlockingRecv error: recv got EOF from 127.0.0.1:49856 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 03:37:00.078173 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:00.074639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43300 (local address 127.27.40.189:35331)
0809 03:37:00.074672 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.074673 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.074890 (+ 217us) server_negotiation.cc:407] Connection header received
0809 03:37:00.074992 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.074993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.075031 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.075076 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:00.075285 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.076012 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.077172 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.077478 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.077518 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:00.077785 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:00.077912 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:00.077962 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:00.078073 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 03:37:00.078091 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 03:37:00.081322 28287 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:00.080536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49870 (local address 127.27.40.188:35647)
0809 03:37:00.080572 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.080574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.080793 (+ 219us) server_negotiation.cc:407] Connection header received
0809 03:37:00.080896 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.080897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.080915 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.080944 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:00.081275 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49870: BlockingRecv error: recv got EOF from 127.0.0.1:49870 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:37:00.086730 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:00.083541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.27.40.189:35331)
0809 03:37:00.083573 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.083574 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.083771 (+ 197us) server_negotiation.cc:407] Connection header received
0809 03:37:00.083860 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.083861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.083870 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.083897 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:00.084064 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.084824 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.085950 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.086243 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.086285 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:00.086478 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:00.086510 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:00.086540 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:00.086640 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 03:37:00.086661 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 03:37:00.090018 28287 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:00.088984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49886 (local address 127.27.40.188:35647)
0809 03:37:00.089019 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.089021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.089427 (+ 406us) server_negotiation.cc:407] Connection header received
0809 03:37:00.089521 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.089522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.089625 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.089657 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:00.089972 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49886: BlockingRecv error: recv got EOF from 127.0.0.1:49886 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 03:37:00.096361 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:00.093032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43322 (local address 127.27.40.189:35331)
0809 03:37:00.093075 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.093077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.093304 (+ 227us) server_negotiation.cc:407] Connection header received
0809 03:37:00.093395 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.093396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.093404 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.093435 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:00.093683 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.094428 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.095578 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.095866 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.095905 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:00.096096 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:00.096140 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:00.096170 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:00.096276 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 03:37:00.096293 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 03:37:00.099270 28287 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:00.098381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49896 (local address 127.27.40.188:35647)
0809 03:37:00.098414 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.098416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.098760 (+ 344us) server_negotiation.cc:407] Connection header received
0809 03:37:00.098854 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.098855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.098865 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.098894 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:00.099221 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49896: BlockingRecv error: recv got EOF from 127.0.0.1:49896 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 03:37:00.107466 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:00.104291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43330 (local address 127.27.40.189:35331)
0809 03:37:00.104327 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.104329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.104573 (+ 244us) server_negotiation.cc:407] Connection header received
0809 03:37:00.104663 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.104664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.104673 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.104700 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:00.104858 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.105611 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.106691 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.106977 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.107016 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:00.107217 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:00.107250 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:00.107278 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:00.107378 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 03:37:00.107395 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 03:37:00.110435 28287 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:00.109665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49908 (local address 127.27.40.188:35647)
0809 03:37:00.109704 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.109706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.109930 (+ 224us) server_negotiation.cc:407] Connection header received
0809 03:37:00.110018 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.110019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.110029 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.110057 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:00.110384 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49908: BlockingRecv error: recv got EOF from 127.0.0.1:49908 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 03:37:00.123250 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:00.119687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43346 (local address 127.27.40.189:35331)
0809 03:37:00.119728 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.119730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.120013 (+ 283us) server_negotiation.cc:407] Connection header received
0809 03:37:00.120121 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.120122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.120147 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.120177 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:00.120370 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.121115 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.122359 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.122684 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.122727 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:00.122955 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:00.122988 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:00.123016 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:00.123119 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 03:37:00.123175 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 03:37:00.126535 28287 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:00.125700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49912 (local address 127.27.40.188:35647)
0809 03:37:00.125741 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.125743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.125928 (+ 185us) server_negotiation.cc:407] Connection header received
0809 03:37:00.126030 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.126031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.126043 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.126074 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:00.126465 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49912: BlockingRecv error: recv got EOF from 127.0.0.1:49912 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 03:37:00.147188 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:00.143832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43358 (local address 127.27.40.189:35331)
0809 03:37:00.143871 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.143873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.144127 (+ 254us) server_negotiation.cc:407] Connection header received
0809 03:37:00.144244 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.144246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.144256 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.144284 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:00.144462 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.145175 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.146367 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.146665 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.146709 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:00.146914 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:00.146947 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:00.146974 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:00.147077 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 03:37:00.147095 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 03:37:00.150343 28287 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:00.149512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49916 (local address 127.27.40.188:35647)
0809 03:37:00.149593 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.149595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.149777 (+ 182us) server_negotiation.cc:407] Connection header received
0809 03:37:00.149873 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.149874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.149885 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.149915 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:00.150290 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49916: BlockingRecv error: recv got EOF from 127.0.0.1:49916 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 03:37:00.187258 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:00.183848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43372 (local address 127.27.40.189:35331)
0809 03:37:00.183899 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.183901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.184205 (+ 304us) server_negotiation.cc:407] Connection header received
0809 03:37:00.184318 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.184319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.184333 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.184364 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:00.184561 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.185343 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.186487 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.186784 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.186822 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:00.186995 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:00.187027 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:00.187051 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:00.187156 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 03:37:00.187174 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 03:37:00.190488 28287 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:00.189620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49920 (local address 127.27.40.188:35647)
0809 03:37:00.189661 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.189663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.189909 (+ 246us) server_negotiation.cc:407] Connection header received
0809 03:37:00.189998 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.189999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.190010 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.190038 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:00.190435 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49920: BlockingRecv error: recv got EOF from 127.0.0.1:49920 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 03:37:00.258939 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:00.255920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43382 (local address 127.27.40.189:35331)
0809 03:37:00.255985 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.255988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.256132 (+ 144us) server_negotiation.cc:407] Connection header received
0809 03:37:00.256169 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.256170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.256184 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.256257 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:00.256391 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.257259 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.258089 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.258438 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.258535 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:00.258664 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:00.258705 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:00.258768 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:00.258819 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 03:37:00.258841 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 03:37:00.261387 28287 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:00.260798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49922 (local address 127.27.40.188:35647)
0809 03:37:00.260856 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.260859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.260869 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:37:00.260940 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.260941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.260953 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.260997 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:00.261330 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49922: BlockingRecv error: recv got EOF from 127.0.0.1:49922 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 03:37:00.393452 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:00.390807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43384 (local address 127.27.40.189:35331)
0809 03:37:00.390892 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.390895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.391066 (+ 171us) server_negotiation.cc:407] Connection header received
0809 03:37:00.391140 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.391142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.391154 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.391219 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:00.391341 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.391821 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.392686 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.392873 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.392938 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:00.393070 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:00.393123 (+ 53us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:00.393242 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:00.393314 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 03:37:00.393339 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 03:37:00.396114 28287 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:00.395459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49928 (local address 127.27.40.188:35647)
0809 03:37:00.395536 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.395538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.395576 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:37:00.395637 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.395638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.395650 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.395725 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:00.396052 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49928: BlockingRecv error: recv got EOF from 127.0.0.1:49928 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 03:37:00.522970 28354 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad06b9de24c2461f8490a89f37ce6185" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad06b9de24c2461f8490a89f37ce6185" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 41487 } } peers { permanent_uuid: "96ba2f5ea2f6495c8fe5817699e301d5" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 35331 } } peers { permanent_uuid: "f6228cf27d864568ba24c87466ffb5a4" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 35647 } } }
I20260809 03:37:00.523170 28354 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad06b9de24c2461f8490a89f37ce6185 [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:00.655865 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:00.653472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43400 (local address 127.27.40.189:35331)
0809 03:37:00.653627 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.653629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.653641 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:37:00.653673 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.653681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.653693 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.653769 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:00.653919 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.654454 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.655193 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:00.655391 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:00.655458 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:00.655590 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:00.655621 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:00.655665 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:00.655724 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 03:37:00.655743 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
W20260809 03:37:00.658272 28287 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:00.657714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49944 (local address 127.27.40.188:35647)
0809 03:37:00.657786 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:00.657788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:00.657797 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:37:00.657827 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:00.657828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:00.657839 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:00.657891 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:00.658216 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49944: BlockingRecv error: recv got EOF from 127.0.0.1:49944 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:37:00.993211 28312 catalog_manager.cc:1259] Loaded cluster ID: dc69bf1e45df4627ae721ebefb5820b5
I20260809 03:37:00.993475 28312 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4: loading cluster ID for follower catalog manager: success
I20260809 03:37:00.996585 28312 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4: acquiring CA information for follower catalog manager: success
I20260809 03:37:00.997768 28312 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f6228cf27d864568ba24c87466ffb5a4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:37:01.184532 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:01.175487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43406 (local address 127.27.40.189:35331)
0809 03:37:01.176117 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:01.176120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:01.176192 (+ 72us) server_negotiation.cc:407] Connection header received
0809 03:37:01.176403 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:01.176404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:01.176421 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:01.176889 (+ 468us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:01.177262 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:01.179092 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:01.180709 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:01.182248 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:01.182678 (+ 430us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:01.183187 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:01.183272 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:01.183604 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:01.183887 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 03:37:01.183962 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":327,"threads_started":1}
I20260809 03:37:01.202158 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:01.191289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49958 (local address 127.27.40.188:35647)
0809 03:37:01.191648 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:01.191664 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:01.191938 (+ 274us) server_negotiation.cc:407] Connection header received
0809 03:37:01.192446 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:01.192448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:01.192591 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:01.192850 (+ 259us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:01.193337 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:01.195158 (+ 1821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:01.199616 (+ 4458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:01.200015 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:01.200108 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:01.200529 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:01.200788 (+ 259us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:01.201001 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:01.201406 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 03:37:01.201467 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":267,"threads_started":1}
I20260809 03:37:01.224466 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:01.210259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49968 (local address 127.27.40.188:35647)
0809 03:37:01.210848 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:01.210850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:01.210872 (+ 22us) server_negotiation.cc:407] Connection header received
0809 03:37:01.210913 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:01.210914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:01.210938 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:01.211611 (+ 673us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:01.211797 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:01.214386 (+ 2589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:01.219571 (+ 5185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:01.220593 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:01.221874 (+ 1281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:01.221910 (+ 36us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:01.222060 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:01.223545 (+ 1485us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:01.223749 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 03:37:01.223850 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546}
I20260809 03:37:01.227376 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:01.209918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55032 (local address 127.27.40.190:41487)
0809 03:37:01.210192 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:01.210204 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:01.210919 (+ 715us) server_negotiation.cc:407] Connection header received
0809 03:37:01.211183 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:01.211184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:01.211293 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:01.211589 (+ 296us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:01.212213 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:01.214725 (+ 2512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:01.219941 (+ 5216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:01.221218 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:01.221524 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:01.224473 (+ 2949us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:01.225085 (+ 612us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:01.225410 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:01.226882 (+ 1472us) server_negotiation.cc:299] Negotiation successful
0809 03:37:01.227015 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
I20260809 03:37:01.228303 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:01.210928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43420 (local address 127.27.40.189:35331)
0809 03:37:01.211105 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:01.211117 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:01.211349 (+ 232us) server_negotiation.cc:407] Connection header received
0809 03:37:01.212520 (+ 1171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:01.212529 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:01.212616 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:01.212791 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:01.213821 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:01.216131 (+ 2310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:01.225386 (+ 9255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:01.226408 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:01.226566 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:01.227588 (+ 1022us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:01.227786 (+ 198us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:01.227847 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:01.228104 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 03:37:01.228145 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260809 03:37:01.233937 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 27955
I20260809 03:37:01.296578 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 28066
I20260809 03:37:01.347741 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 28188
2026-08-09T03:37:01Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3205 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:37:01 dist-test-slave-gkw7 krb5kdc[28562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:37:01 dist-test-slave-gkw7 krb5kdc[28562](info): set up 2 sockets
Aug 09 03:37:01 dist-test-slave-gkw7 krb5kdc[28562](info): commencing operation
krb5kdc: starting...
W20260809 03:37:02.617652 27810 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/test-user.keytab.
Aug 09 03:37:02 dist-test-slave-gkw7 krb5kdc[28562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:37:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:37:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.40.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
Entry for principal kudu/127.27.40.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab.
I20260809 03:37:02.744050 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:44923
--webserver_interface=127.27.40.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:44923,127.27.40.189:42015,127.27.40.188:34371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:37:02.826700 28581 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:37:02.826901 28581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:02.826918 28581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:02.826938 28581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:02.828527 28581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:37:02.828588 28581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:02.828615 28581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:02.828644 28581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:37:02.828661 28581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:37:02.830247 28581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:44923,127.27.40.189:42015,127.27.40.188:34371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.190.keytab
--principal=kudu/127.27.40.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.190:44923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.40.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:37:02.830444 28581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:02.830621 28581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:02.835417 28589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:02.835433 28591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:02.835422 28587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:02.835733 28581 server_base.cc:1034] running on GCE node
I20260809 03:37:02.835990 28581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:02.836283 28581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:02.837452 28581 hybrid_clock.cc:648] HybridClock initialized: now 1786246622837445 us; error 36 us; skew 500 ppm
Aug 09 03:37:02 dist-test-slave-gkw7 krb5kdc[28562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246622, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:02.840029 28581 init.cc:377] Logged in from keytab as kudu/127.27.40.190@KRBTEST.COM (short username kudu)
I20260809 03:37:02.840505 28581 webserver.cc:460] Webserver started at http://127.27.40.190:33553/ using document root <none> and password file <none>
I20260809 03:37:02.840770 28581 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:02.840840 28581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:02.840991 28581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:37:02.842149 28581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/data/instance:
uuid: "12df6e9e434c49a49f0e961705387c25"
format_stamp: "Formatted at 2026-08-09 03:37:02 on dist-test-slave-gkw7"
server_key: "ddae355128048799075aeb6715c493ad"
server_key_iv: "a8ec1b3b2ffb83e9a340d7cbd2281249"
server_key_version: "encryptionkey@0"
I20260809 03:37:02.842502 28581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/wal/instance:
uuid: "12df6e9e434c49a49f0e961705387c25"
format_stamp: "Formatted at 2026-08-09 03:37:02 on dist-test-slave-gkw7"
server_key: "ddae355128048799075aeb6715c493ad"
server_key_iv: "a8ec1b3b2ffb83e9a340d7cbd2281249"
server_key_version: "encryptionkey@0"
I20260809 03:37:02.843854 28581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 03:37:02.844591 28600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:02.844818 28581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:37:02.844934 28581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/wal
uuid: "12df6e9e434c49a49f0e961705387c25"
format_stamp: "Formatted at 2026-08-09 03:37:02 on dist-test-slave-gkw7"
server_key: "ddae355128048799075aeb6715c493ad"
server_key_iv: "a8ec1b3b2ffb83e9a340d7cbd2281249"
server_key_version: "encryptionkey@0"
I20260809 03:37:02.845010 28581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:02.859786 28581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:02.862198 28581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:02.862370 28581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:02.866462 28581 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.190:44923
I20260809 03:37:02.866498 28669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.190:44923 every 8 connection(s)
I20260809 03:37:02.866868 28581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/data/info.pb
I20260809 03:37:02.867884 28670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:02.869283 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 28581
I20260809 03:37:02.869402 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-0/wal/instance
I20260809 03:37:02.869663 27810 external_mini_cluster.cc:1428] Setting key f7841f7b022eadb32d70c14d3feeb987
I20260809 03:37:02.869680 28670 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:37:02 dist-test-slave-gkw7 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246622, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.40.189@KRBTEST.COM; defaulting to no policy
I20260809 03:37:02.880686 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:02.870520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:44923 (local address 127.0.0.1:56922)
0809 03:37:02.870710 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:02.870746 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:02.871229 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:02.871548 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:02.871551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:02.871923 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:02.872159 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:02.872165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:02.873488 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:02.873488 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:02.874277 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:02.874280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:02.874997 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:02.874998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:02.875053 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:02.875582 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:02.875589 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:02.877194 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:02.879272 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:02.879279 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:02.879280 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:02.879458 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:02.879663 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:02.879663 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:02.879664 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:02.879738 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:02.880223 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:02.880229 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:02.880285 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 03:37:02.880360 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 03:37:02.880477 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":108,"threads_started":1}
I20260809 03:37:02.880697 28675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:02.870543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56922 (local address 127.27.40.190:44923)
0809 03:37:02.870726 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:02.870735 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:02.870884 (+ 149us) server_negotiation.cc:407] Connection header received
0809 03:37:02.871373 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:02.871376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:02.871399 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:02.871517 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:02.872262 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:02.873393 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:02.874374 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:02.874883 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:02.875013 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:02.877288 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:02.877297 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:02.877297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:02.877304 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:02.879200 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:02.879528 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:02.879530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:02.879530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:02.879606 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:02.879818 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:02.879819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:02.879819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:02.880139 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:02.880231 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:02.880426 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 03:37:02.880547 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
I20260809 03:37:02.881587 28670 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.189@KRBTEST.COM" created.
I20260809 03:37:02.881986 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:02.881873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:42015 (local address 127.0.0.1:54034)
0809 03:37:02.881920 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:02.881968 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:42015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 03:37:02.882117 28608 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.40.189:42015: connect: Connection refused (error 111)
I20260809 03:37:02.882629 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:02.882513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:42015 (local address 127.0.0.1:54046)
0809 03:37:02.882567 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:02.882599 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:42015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 03:37:02.882761 28670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.189:42015: Network error: Client connection negotiation failed: client connection to 127.27.40.189:42015: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.189.keytab.
Entry for principal kudu/127.27.40.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.189.keytab.
I20260809 03:37:02.896786 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.189:42015
--webserver_interface=127.27.40.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.189.keytab
--principal=kudu/127.27.40.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:44923,127.27.40.189:42015,127.27.40.188:34371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:37:02.931993 28670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } attempt: 1
I20260809 03:37:02.932507 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:02.932369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:42015 (local address 127.0.0.1:54058)
0809 03:37:02.932435 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:02.932476 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:42015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 03:37:02.932898 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:02.932796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:42015 (local address 127.0.0.1:54070)
0809 03:37:02.932845 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:02.932878 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:42015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 03:37:02.932991 28670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.189:42015: Network error: Client connection negotiation failed: client connection to 127.27.40.189:42015: connect: Connection refused (error 111)
W20260809 03:37:02.980415 28679 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:37:02.980665 28679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:02.980697 28679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:02.980739 28679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:02.981997 28679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:37:02.982057 28679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:02.982092 28679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:02.982125 28679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:37:02.982151 28679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:37:02.983675 28679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:44923,127.27.40.189:42015,127.27.40.188:34371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.189.keytab
--principal=kudu/127.27.40.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.189:42015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.40.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:37:02.983970 28679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:02.984205 28679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:02.989167 28689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:02.989359 28679 server_base.cc:1034] running on GCE node
W20260809 03:37:02.989176 28688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:02.989223 28691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:02.989745 28679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:02.990048 28679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:02.991200 28679 hybrid_clock.cc:648] HybridClock initialized: now 1786246622991181 us; error 39 us; skew 500 ppm
Aug 09 03:37:02 dist-test-slave-gkw7 krb5kdc[28562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246622, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:02.993810 28679 init.cc:377] Logged in from keytab as kudu/127.27.40.189@KRBTEST.COM (short username kudu)
I20260809 03:37:02.994303 28679 webserver.cc:460] Webserver started at http://127.27.40.189:36315/ using document root <none> and password file <none>
I20260809 03:37:02.994547 28679 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:02.994596 28679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:02.994766 28679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:37:02.995814 28679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/data/instance:
uuid: "29fc40e23f0549d69e9dde39c553b8ee"
format_stamp: "Formatted at 2026-08-09 03:37:02 on dist-test-slave-gkw7"
server_key: "156996bbc9ea81f47fd7502d954a9154"
server_key_iv: "cc4aa7ba64ddeae3480ca94a577ade0c"
server_key_version: "encryptionkey@0"
I20260809 03:37:02.996196 28679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/wal/instance:
uuid: "29fc40e23f0549d69e9dde39c553b8ee"
format_stamp: "Formatted at 2026-08-09 03:37:02 on dist-test-slave-gkw7"
server_key: "156996bbc9ea81f47fd7502d954a9154"
server_key_iv: "cc4aa7ba64ddeae3480ca94a577ade0c"
server_key_version: "encryptionkey@0"
I20260809 03:37:02.997620 28679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 03:37:02.998490 28699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:02.998766 28679 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:37:02.998879 28679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/wal
uuid: "29fc40e23f0549d69e9dde39c553b8ee"
format_stamp: "Formatted at 2026-08-09 03:37:02 on dist-test-slave-gkw7"
server_key: "156996bbc9ea81f47fd7502d954a9154"
server_key_iv: "cc4aa7ba64ddeae3480ca94a577ade0c"
server_key_version: "encryptionkey@0"
I20260809 03:37:02.998955 28679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:03.001139 28670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } attempt: 2
I20260809 03:37:03.001636 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.001457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:42015 (local address 127.0.0.1:54080)
0809 03:37:03.001519 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.001606 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:42015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 03:37:03.001974 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.001873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:42015 (local address 127.0.0.1:54084)
0809 03:37:03.001919 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.001944 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.189:42015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 03:37:03.002089 28670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.189:42015: Network error: Client connection negotiation failed: client connection to 127.27.40.189:42015: connect: Connection refused (error 111)
I20260809 03:37:03.027454 28679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:03.030016 28679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:03.030200 28679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:03.034621 28679 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.189:42015
I20260809 03:37:03.034679 28783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.189:42015 every 8 connection(s)
I20260809 03:37:03.035045 28679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/data/info.pb
I20260809 03:37:03.036083 28784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:03.037755 28784 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:37:03.042837 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 28679
I20260809 03:37:03.042933 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-1/wal/instance
I20260809 03:37:03.043144 27810 external_mini_cluster.cc:1428] Setting key 3f43bc91e3c0abde55fd7a07bf60bb7e
Aug 09 03:37:03 dist-test-slave-gkw7 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246622, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.189@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:37:03.047847 28675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.038451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56924 (local address 127.27.40.190:44923)
0809 03:37:03.038519 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.038521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.038726 (+ 205us) server_negotiation.cc:407] Connection header received
0809 03:37:03.039283 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.039284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.039302 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.039353 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.039974 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.040492 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.041604 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.041840 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.041932 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.044547 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:03.044548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:03.044548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:03.044555 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:03.046303 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.046710 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.046712 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.046712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.046781 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.047031 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.047031 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.047032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.047210 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:03.047278 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:03.047565 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 03:37:03.047681 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 03:37:03.047904 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.038432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:44923 (local address 127.0.0.1:56924)
0809 03:37:03.038591 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.038626 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:03.039160 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:03.039427 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:03.039430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:03.039679 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:03.039872 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:03.039876 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.040598 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.040599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:03.041475 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:03.041477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.041958 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.041960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:03.042112 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.042571 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:03.042580 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:03.044438 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:03.046393 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:03.046401 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:03.046402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:03.046631 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:03.046852 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:03.046852 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:03.046853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:03.046963 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:03.047277 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:03.047282 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:03.047415 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 03:37:03.047500 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 03:37:03.047695 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
I20260809 03:37:03.048509 28784 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.40.188@KRBTEST.COM; defaulting to no policy
Aug 09 03:37:03 dist-test-slave-gkw7 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246622, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.189@KRBTEST.COM for kudu/127.27.40.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.40.188@KRBTEST.COM" created.
I20260809 03:37:03.057099 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.048810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:42015 (local address 127.0.0.1:54086)
0809 03:37:03.048847 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.048857 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:03.048902 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:03.049208 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:03.049209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:03.049323 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:03.049375 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:03.049376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.050814 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.050815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:03.051322 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:03.051324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.052002 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.052003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:03.052056 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.052465 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:03.052469 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:03.053980 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:03.055822 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:03.055823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:03.055824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:03.055996 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:03.056278 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:03.056279 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:03.056280 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:03.056386 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:03.056694 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:03.056695 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:03.056744 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 03:37:03.056827 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 03:37:03.056950 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 03:37:03.057108 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.048883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54086 (local address 127.27.40.189:42015)
0809 03:37:03.049052 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.049057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.049089 (+ 32us) server_negotiation.cc:407] Connection header received
0809 03:37:03.049110 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.049113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.049128 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.049206 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.049456 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.050717 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.051412 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.051914 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.051990 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.054089 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:03.054096 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:03.054097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:03.054103 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:03.055743 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.056105 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.056108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.056109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.056175 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.056474 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.056475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.056475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.056622 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:03.056689 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:03.056850 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 03:37:03.056997 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260809 03:37:03.058068 28784 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:37:03.058418 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.058304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:34371 (local address 127.0.0.1:51682)
0809 03:37:03.058355 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.058398 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 03:37:03.058580 28701 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.40.188:34371: connect: Connection refused (error 111)
I20260809 03:37:03.059007 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.058916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:34371 (local address 127.0.0.1:51694)
0809 03:37:03.058967 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.058991 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 03:37:03.059126 28784 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.188:34371: Network error: Client connection negotiation failed: client connection to 127.27.40.188:34371: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.40.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.188.keytab.
Entry for principal kudu/127.27.40.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.188.keytab.
I20260809 03:37:03.072902 27810 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcMSfqS/build/release/bin/kudu
/tmp/dist-test-taskcMSfqS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.188:34371
--webserver_interface=127.27.40.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.188.keytab
--principal=kudu/127.27.40.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.40.148:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.40.190:44923,127.27.40.189:42015,127.27.40.188:34371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:37:03.093266 28670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } attempt: 3
Aug 09 03:37:03 dist-test-slave-gkw7 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246622, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for kudu/127.27.40.189@KRBTEST.COM
I20260809 03:37:03.100602 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.093713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:42015 (local address 127.0.0.1:54090)
0809 03:37:03.093787 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.093801 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:03.093861 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:03.094032 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:03.094033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:03.094235 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:03.094305 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:03.094308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.095036 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.095037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:03.095590 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:03.095591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.096010 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.096011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:03.096114 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.096595 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:03.096608 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:03.097976 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:03.099674 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:03.099675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:03.099676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:03.099841 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:03.100041 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:03.100041 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:03.100042 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:03.100095 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:03.100353 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:03.100353 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:03.100393 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 03:37:03.100444 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 03:37:03.100518 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 03:37:03.100673 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.093788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54090 (local address 127.27.40.189:42015)
0809 03:37:03.093862 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.093864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.093875 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:37:03.093925 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.093925 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.093936 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.094027 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.094401 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.094930 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.095684 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.095918 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.096009 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.098076 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:03.098077 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:03.098077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:03.098086 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:03.099580 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.099909 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.099911 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.099911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.099975 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.100155 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.100156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.100156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.100290 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:03.100358 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:03.100489 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 03:37:03.100573 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:37:03.101022 28670 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:37:03.101392 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.101280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:34371 (local address 127.0.0.1:51702)
0809 03:37:03.101338 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.101368 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 03:37:03.102108 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.102003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:34371 (local address 127.0.0.1:51712)
0809 03:37:03.102054 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.102078 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 03:37:03.102211 28670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.188:34371: Network error: Client connection negotiation failed: client connection to 127.27.40.188:34371: connect: Connection refused (error 111)
I20260809 03:37:03.108371 28784 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } attempt: 1
I20260809 03:37:03.108820 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.108693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:34371 (local address 127.0.0.1:51724)
0809 03:37:03.108755 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.108794 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 03:37:03.109165 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.109071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:34371 (local address 127.0.0.1:51736)
0809 03:37:03.109116 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.109146 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 03:37:03.109300 28784 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.188:34371: Network error: Client connection negotiation failed: client connection to 127.27.40.188:34371: connect: Connection refused (error 111)
I20260809 03:37:03.133374 28670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } attempt: 1
I20260809 03:37:03.133883 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.133745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:34371 (local address 127.0.0.1:51752)
0809 03:37:03.133812 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.133855 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 03:37:03.134277 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.134179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:34371 (local address 127.0.0.1:51758)
0809 03:37:03.134229 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.134260 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 03:37:03.134366 28670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.188:34371: Network error: Client connection negotiation failed: client connection to 127.27.40.188:34371: connect: Connection refused (error 111)
W20260809 03:37:03.157845 28796 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:37:03.158056 28796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:03.158073 28796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:03.158116 28796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:03.159232 28796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:37:03.159273 28796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:03.159287 28796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:03.159300 28796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:37:03.159312 28796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:37:03.160758 28796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.40.148:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.40.190:44923,127.27.40.189:42015,127.27.40.188:34371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/krb5kdc/kudu_127.27.40.188.keytab
--principal=kudu/127.27.40.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.40.188:34371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.40.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 02:27:49 UTC on ccycloud.quasar-uxsfuz.root.comops.site
build id 4575768
I20260809 03:37:03.161007 28796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:03.161268 28796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:03.166332 28808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:03.166353 28803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:03.166311 28804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:03.166543 28796 server_base.cc:1034] running on GCE node
I20260809 03:37:03.166774 28796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:03.167070 28796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:03.168255 28796 hybrid_clock.cc:648] HybridClock initialized: now 1786246623168251 us; error 37 us; skew 500 ppm
Aug 09 03:37:03 dist-test-slave-gkw7 krb5kdc[28562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246623, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:03.170784 28796 init.cc:377] Logged in from keytab as kudu/127.27.40.188@KRBTEST.COM (short username kudu)
I20260809 03:37:03.171293 28796 webserver.cc:460] Webserver started at http://127.27.40.188:39563/ using document root <none> and password file <none>
I20260809 03:37:03.171520 28796 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:03.171587 28796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:03.171744 28796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:37:03.172811 28796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/data/instance:
uuid: "d14549caa7c742e6afd21878ebe4aa90"
format_stamp: "Formatted at 2026-08-09 03:37:03 on dist-test-slave-gkw7"
server_key: "fbb83cd1c0c12198511f63ccbadbfa2b"
server_key_iv: "db2b4d473ae3d009920739e91c3ad58f"
server_key_version: "encryptionkey@0"
I20260809 03:37:03.173174 28796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/wal/instance:
uuid: "d14549caa7c742e6afd21878ebe4aa90"
format_stamp: "Formatted at 2026-08-09 03:37:03 on dist-test-slave-gkw7"
server_key: "fbb83cd1c0c12198511f63ccbadbfa2b"
server_key_iv: "db2b4d473ae3d009920739e91c3ad58f"
server_key_version: "encryptionkey@0"
I20260809 03:37:03.174597 28796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 03:37:03.175377 28818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:03.175561 28796 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:37:03.175660 28796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/data,/tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/wal
uuid: "d14549caa7c742e6afd21878ebe4aa90"
format_stamp: "Formatted at 2026-08-09 03:37:03 on dist-test-slave-gkw7"
server_key: "fbb83cd1c0c12198511f63ccbadbfa2b"
server_key_iv: "db2b4d473ae3d009920739e91c3ad58f"
server_key_version: "encryptionkey@0"
I20260809 03:37:03.175738 28796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:03.177476 28784 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } attempt: 2
I20260809 03:37:03.177991 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.177846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:34371 (local address 127.0.0.1:51764)
0809 03:37:03.177916 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.177956 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 03:37:03.178349 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.178239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:34371 (local address 127.0.0.1:51768)
0809 03:37:03.178291 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.178324 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.40.188:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 03:37:03.178473 28784 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.40.188:34371: Network error: Client connection negotiation failed: client connection to 127.27.40.188:34371: connect: Connection refused (error 111)
I20260809 03:37:03.187176 28796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:03.189211 28796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:03.189354 28796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:03.193310 28796 rpc_server.cc:307] RPC server started. Bound to: 127.27.40.188:34371
I20260809 03:37:03.193384 28897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.40.188:34371 every 8 connection(s)
I20260809 03:37:03.193774 28796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/data/info.pb
I20260809 03:37:03.194747 28898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:03.196239 28898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:37:03.198251 27810 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcMSfqS/build/release/bin/kudu as pid 28796
I20260809 03:37:03.198379 27810 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcMSfqS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246339702443-27810-0/minicluster-data/master-2/wal/instance
I20260809 03:37:03.198604 27810 external_mini_cluster.cc:1428] Setting key d19216fbeaeb0bb27b3549e690f1d001
Aug 09 03:37:03 dist-test-slave-gkw7 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246623, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.188@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
Aug 09 03:37:03 dist-test-slave-gkw7 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.188@KRBTEST.COM
I20260809 03:37:03.206223 28675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.196914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56928 (local address 127.27.40.190:44923)
0809 03:37:03.196965 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.196966 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.197247 (+ 281us) server_negotiation.cc:407] Connection header received
0809 03:37:03.197702 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.197703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.197733 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.197800 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.198445 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.199199 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.200393 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.200587 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.200641 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.203240 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:03.203242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:03.203242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:03.203249 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:03.204692 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.205153 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.205154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.205154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.205209 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.205524 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.205547 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.205547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.205702 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:03.205757 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:03.206086 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 03:37:03.206158 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 03:37:03.206517 28900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.196880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.190:44923 (local address 127.0.0.1:56928)
0809 03:37:03.197093 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.197127 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:03.197617 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:03.197882 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:03.197886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:03.198129 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:03.198320 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:03.198324 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.199288 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.199289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:03.200305 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:03.200308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.200683 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.200684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:03.200810 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.201352 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:03.201360 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:03.203151 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:03.204769 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:03.204776 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:03.204778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:03.205072 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:03.205291 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:03.205291 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:03.205292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:03.205429 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:03.205774 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:03.205782 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:03.205943 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 03:37:03.206065 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 03:37:03.206263 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":105,"threads_started":1}
I20260809 03:37:03.207144 28898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:37:03.207978 28902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.198972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51770 (local address 127.27.40.188:34371)
0809 03:37:03.199142 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.199147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.199185 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:37:03.199230 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.199231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.199272 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.199356 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.199918 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.200893 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.201811 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.202196 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.202252 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.204616 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:03.204625 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:03.204626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:03.204636 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:03.206513 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.207098 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.207101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.207101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.207168 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.207401 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.207401 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.207402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.207580 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:03.207715 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:03.207795 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 03:37:03.207869 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":97,"threads_started":1}
I20260809 03:37:03.209506 28670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } attempt: 2
Aug 09 03:37:03 dist-test-slave-gkw7 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246623, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.188@KRBTEST.COM for kudu/127.27.40.189@KRBTEST.COM
Aug 09 03:37:03 dist-test-slave-gkw7 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.190@KRBTEST.COM
I20260809 03:37:03.214581 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.207445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54096 (local address 127.27.40.189:42015)
0809 03:37:03.207485 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.207487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.207494 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:37:03.207525 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.207526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.207534 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.207577 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.207851 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.208540 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.209369 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.209763 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.209828 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.211648 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:03.211649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:03.211650 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:03.211657 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:03.213313 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.213730 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.213731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.213732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.213791 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.214025 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.214026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.214026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.214174 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:03.214267 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:03.214423 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 03:37:03.214493 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 03:37:03.214581 28900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.207389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.189:42015 (local address 127.0.0.1:54096)
0809 03:37:03.207429 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.207438 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:03.207471 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:03.207590 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:03.207591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:03.207732 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:03.207766 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:03.207768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.208622 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.208623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:03.209277 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:03.209279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.209826 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.209827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:03.209892 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.210415 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:03.210420 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:03.211581 (+ 1161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:03.213397 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:03.213398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:03.213398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:03.213650 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:03.213858 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:03.213858 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:03.213859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:03.213952 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:03.214261 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:03.214262 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:03.214305 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 03:37:03.214396 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 03:37:03.214485 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
Aug 09 03:37:03 dist-test-slave-gkw7 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246622, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.190@KRBTEST.COM for kudu/127.27.40.188@KRBTEST.COM
I20260809 03:37:03.215423 28898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:37:03.217072 28675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.208775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56944 (local address 127.27.40.190:44923)
0809 03:37:03.208826 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.208828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.208836 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:37:03.208854 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.208855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.208863 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.208926 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.209394 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.210247 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.211192 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.211457 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.211526 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.213926 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:03.213927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:03.213927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:03.213934 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:03.215896 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.216270 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.216271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.216272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.216333 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.216572 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.216573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.216573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.216705 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:03.216859 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:03.216892 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 03:37:03.216964 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 03:37:03 dist-test-slave-gkw7 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246623, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.188@KRBTEST.COM for kudu/127.27.40.188@KRBTEST.COM
I20260809 03:37:03.220166 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.209757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:34371 (local address 127.0.0.1:51772)
0809 03:37:03.209792 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.209803 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:03.209841 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:03.210092 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:03.210093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:03.210265 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:03.210312 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:03.210314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.211162 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.211163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:03.211929 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:03.211932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.213603 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.213604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:03.213682 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.214050 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:03.214055 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:03.215855 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:03.219119 (+ 3264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:03.219120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:03.219120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:03.219327 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:03.219534 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:03.219535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:03.219535 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:03.219610 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:03.219869 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:03.219870 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:03.219913 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 03:37:03.219987 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 03:37:03.220073 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":2}
I20260809 03:37:03.220366 28902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.209904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51772 (local address 127.27.40.188:34371)
0809 03:37:03.209958 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.209960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.209967 (+ 7us) server_negotiation.cc:407] Connection header received
0809 03:37:03.209984 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.209985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.209994 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.210076 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.210369 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.211069 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.212020 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.213485 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.213586 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.215950 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:03.215952 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:03.215952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:03.215958 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:03.219046 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.219402 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.219403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.219404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.219461 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.219667 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.219668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.219668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.219790 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:03.219865 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:03.220031 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 03:37:03.220125 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 03:37:03.222258 28670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25: Bootstrap starting.
I20260809 03:37:03.223003 28900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.215651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:34371 (local address 127.0.0.1:51776)
0809 03:37:03.215796 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.215807 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:03.215846 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:03.216006 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:03.216006 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:03.216171 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:03.216193 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:03.216194 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.217057 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.217065 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:03.217677 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:03.217680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.218157 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.218158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:03.218208 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.218595 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:03.218600 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:03.219841 (+ 1241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:03.221974 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:03.221975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:03.221976 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:03.222142 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:03.222353 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:03.222354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:03.222354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:03.222409 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:03.222677 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:03.222677 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:03.222716 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 03:37:03.222798 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 03:37:03.222893 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260809 03:37:03.223078 28670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:03.223119 28903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.215681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51776 (local address 127.27.40.188:34371)
0809 03:37:03.215837 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.215839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.215848 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:37:03.215896 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.215898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.215908 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.215972 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.216285 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.216992 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.217763 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.218100 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.218157 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.219916 (+ 1759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:03.219917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:03.219918 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:03.219923 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:03.221876 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.222224 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.222225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.222225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.222279 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.222480 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.222481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.222481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.222605 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:03.222683 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:03.222823 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 03:37:03.222943 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
Aug 09 03:37:03 dist-test-slave-gkw7 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.40.189@KRBTEST.COM
I20260809 03:37:03.223390 28670 log.cc:824] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:03.224254 28670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25: No bootstrap required, opened a new log
I20260809 03:37:03.224545 28898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90: Bootstrap starting.
I20260809 03:37:03.225284 28898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:03.225633 28898 log.cc:824] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:03.226013 28670 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } }
I20260809 03:37:03.226181 28670 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12df6e9e434c49a49f0e961705387c25, State: Initialized, Role: FOLLOWER
I20260809 03:37:03.226274 28670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } }
I20260809 03:37:03.226429 28898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90: No bootstrap required, opened a new log
I20260809 03:37:03.226624 28670 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:37:03.227111 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.218008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54102 (local address 127.27.40.189:42015)
0809 03:37:03.218043 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.218044 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.218071 (+ 27us) server_negotiation.cc:407] Connection header received
0809 03:37:03.218091 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.218092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.218101 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.218164 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.218869 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.219573 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.220760 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.221034 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.221104 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.224047 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:03.224048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:03.224048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:03.224057 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:03.225888 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.226286 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.226287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.226287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.226356 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.226573 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.226574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.226574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.226717 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:03.226897 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:03.226935 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 03:37:03.227004 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 03:37:03.228444 28923 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 03:37:03.228466 27810 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 03:37:03.228499 28923 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:37:03.228621 28898 raft_consensus.cc:348] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } }
I20260809 03:37:03.228767 28898 raft_consensus.cc:729] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d14549caa7c742e6afd21878ebe4aa90, State: Initialized, Role: FOLLOWER
I20260809 03:37:03.228838 28898 consensus_queue.cc:260] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } }
I20260809 03:37:03.229095 28898 sys_catalog.cc:564] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:37:03.229406 28638 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "12df6e9e434c49a49f0e961705387c25"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56944
I20260809 03:37:03.229578 28638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 03:37:03.229663 28638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:03.229077 28925 sys_catalog.cc:455] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } } }
I20260809 03:37:03.230819 28925 sys_catalog.cc:458] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:37:03.231060 28638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } }
I20260809 03:37:03.231276 28638 leader_election.cc:288] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [CANDIDATE]: Term 1 election: Requested vote from peers 29fc40e23f0549d69e9dde39c553b8ee (127.27.40.189:42015), d14549caa7c742e6afd21878ebe4aa90 (127.27.40.188:34371)
W20260809 03:37:03.231482 28943 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:37:03.231597 28943 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:37:03.231969 28860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "12df6e9e434c49a49f0e961705387c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d14549caa7c742e6afd21878ebe4aa90"
I20260809 03:37:03.232108 28860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:03.232940 28860 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12df6e9e434c49a49f0e961705387c25 in term 1.
I20260809 03:37:03.232178 28581 master_runner.cc:186] Error getting master registration for 127.27.40.189:42015: OK, instance_id { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" instance_seqno: 1786246623033353 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 03:37:03.234584 28581 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 03:37:03.234769 28605 leader_election.cc:302] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12df6e9e434c49a49f0e961705387c25, d14549caa7c742e6afd21878ebe4aa90; no voters:
I20260809 03:37:03.235052 28947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } } }
I20260809 03:37:03.235270 28947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:37:03.235419 28947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:37:03.235553 28947 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [term 1 LEADER]: Becoming Leader. State: Replica: 12df6e9e434c49a49f0e961705387c25, State: Running, Role: LEADER
I20260809 03:37:03.235759 28947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } }
I20260809 03:37:03.236372 28947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12df6e9e434c49a49f0e961705387c25. Latest consensus state: current_term: 1 leader_uuid: "12df6e9e434c49a49f0e961705387c25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } } }
I20260809 03:37:03.236514 28947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:03.236822 28954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:37:03.237488 28954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:37:03.236397 28796 master_runner.cc:186] Error getting master registration for 127.27.40.189:42015: OK, instance_id { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" instance_seqno: 1786246623033353 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 03:37:03.238615 28796 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 03:37:03.246013 28860 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90 [term 1 FOLLOWER]: Refusing update from remote peer 12df6e9e434c49a49f0e961705387c25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:37:03.246397 28907 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:37:03.247753 28925 sys_catalog.cc:455] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12df6e9e434c49a49f0e961705387c25. Latest consensus state: current_term: 1 leader_uuid: "12df6e9e434c49a49f0e961705387c25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } } }
I20260809 03:37:03.247838 28925 sys_catalog.cc:458] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:37:03.253489 28907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "12df6e9e434c49a49f0e961705387c25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } } }
I20260809 03:37:03.253598 28907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:03.253796 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.238455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54108 (local address 127.27.40.189:42015)
0809 03:37:03.238497 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.238499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.239041 (+ 542us) server_negotiation.cc:407] Connection header received
0809 03:37:03.239180 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.239181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.239193 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.239225 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.240474 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.241228 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.242980 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.243294 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.243342 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.250723 (+ 7381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:03.250725 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:03.250725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:03.250733 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:03.252501 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.252922 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.252923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.252923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.252980 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.253177 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.253178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.253179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.253305 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:03.253374 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:03.253620 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 03:37:03.253698 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 03:37:03.254168 28925 sys_catalog.cc:455] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "12df6e9e434c49a49f0e961705387c25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } } }
I20260809 03:37:03.254248 28925 sys_catalog.cc:458] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:37:03.255548 28958 mvcc.cc:205] Tried to move back new op lower bound from 7316466168811638784 to 7316466168775159808. Current Snapshot: MvccSnapshot[applied={T|T < 7316466168811638784}]
I20260809 03:37:03.255892 28961 mvcc.cc:205] Tried to move back new op lower bound from 7316466168811638784 to 7316466168775159808. Current Snapshot: MvccSnapshot[applied={T|T < 7316466168811638784}]
I20260809 03:37:03.257133 28903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.238953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51788 (local address 127.27.40.188:34371)
0809 03:37:03.238988 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.238990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.245792 (+ 6802us) server_negotiation.cc:407] Connection header received
0809 03:37:03.245889 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.245890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.245911 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.245941 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.246804 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.247540 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.248687 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.249044 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.249083 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.251008 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:03.251009 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:03.251009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:03.251016 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:03.252767 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.254783 (+ 2016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.254784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.254784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.254862 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.256348 (+ 1486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.256349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.256350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.256485 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:03.256581 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:03.256958 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 03:37:03.257043 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:37:03.259451 28954 catalog_manager.cc:1347] Generated new cluster ID: 2c104ce2fa3f4064bb7c039b66663324
I20260809 03:37:03.259500 28954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:37:03.262174 28675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.238733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56958 (local address 127.27.40.190:44923)
0809 03:37:03.238782 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.238784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.249468 (+ 10684us) server_negotiation.cc:407] Connection header received
0809 03:37:03.249645 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.249646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.249656 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.249685 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.255311 (+ 5626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.256045 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.257141 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.257444 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.257482 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.259211 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:03.259212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:03.259212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:03.259219 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:03.260930 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.261314 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.261316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.261316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.261376 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.261601 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.261602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.261602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.261724 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:03.261817 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:03.262012 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 03:37:03.262086 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 03:37:03.269619 28784 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } attempt: 3
Aug 09 03:37:03 dist-test-slave-gkw7 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246622, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.40.189@KRBTEST.COM for kudu/127.27.40.188@KRBTEST.COM
I20260809 03:37:03.279109 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.269865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.40.188:34371 (local address 127.0.0.1:51800)
0809 03:37:03.269968 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:03.269978 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:03.270014 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:03.270227 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:03.270228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:03.270387 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:03.270430 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:03.270432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.271353 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.271354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:03.272158 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:03.272161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.272716 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.272717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:03.272791 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.273285 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:03.273291 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:03.274966 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:03.277875 (+ 2909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:03.277876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:03.277876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:03.278097 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:03.278356 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:03.278356 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:03.278357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:03.278432 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:03.278795 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:03.278796 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:03.278856 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 03:37:03.278918 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 03:37:03.279010 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 03:37:03.279742 28903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:03.270079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51800 (local address 127.27.40.188:34371)
0809 03:37:03.270110 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.270112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.270118 (+ 6us) server_negotiation.cc:407] Connection header received
0809 03:37:03.270136 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.270137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.270146 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.270188 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.270505 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.271253 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.272269 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:03.272592 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:03.272649 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:03.275082 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:03.275083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:03.275084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:03.275106 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:03.277768 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.278188 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.278189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.278190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.278274 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:03.278523 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:03.278524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:03.278524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:03.278690 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:03.278750 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:03.279549 (+ 799us) server_negotiation.cc:299] Negotiation successful
0809 03:37:03.279653 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 03:37:03.281615 28784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee: Bootstrap starting.
I20260809 03:37:03.282385 28784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:03.282702 28784 log.cc:824] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:03.284536 28784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee: No bootstrap required, opened a new log
I20260809 03:37:03.286496 28954 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:37:03.286583 28784 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } }
I20260809 03:37:03.286715 28784 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29fc40e23f0549d69e9dde39c553b8ee, State: Initialized, Role: FOLLOWER
I20260809 03:37:03.286788 28784 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } }
I20260809 03:37:03.287122 28784 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:37:03.287197 28954 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:37:03.288371 28740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "12df6e9e434c49a49f0e961705387c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "29fc40e23f0549d69e9dde39c553b8ee"
I20260809 03:37:03.288467 28740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:03.289283 28740 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12df6e9e434c49a49f0e961705387c25 in term 1.
I20260809 03:37:03.289489 28739 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee [term 1 FOLLOWER]: Refusing update from remote peer 12df6e9e434c49a49f0e961705387c25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:37:03.289741 28907 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:37:03.291277 28968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12df6e9e434c49a49f0e961705387c25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } } }
I20260809 03:37:03.292014 28968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:37:03.292119 28968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12df6e9e434c49a49f0e961705387c25. Latest consensus state: current_term: 1 leader_uuid: "12df6e9e434c49a49f0e961705387c25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } } }
I20260809 03:37:03.292218 28968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:37:03.292289 28968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "12df6e9e434c49a49f0e961705387c25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } } }
I20260809 03:37:03.292384 28968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee [sys.catalog]: This master's current role is: FOLLOWER
W20260809 03:37:03.293301 28984 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:37:03.293354 28984 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:37:03.297619 28954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25: Generated new TSK 0
I20260809 03:37:03.297802 28954 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:37:03.298930 28947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "12df6e9e434c49a49f0e961705387c25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12df6e9e434c49a49f0e961705387c25" member_type: VOTER last_known_addr { host: "127.27.40.190" port: 44923 } } peers { permanent_uuid: "29fc40e23f0549d69e9dde39c553b8ee" member_type: VOTER last_known_addr { host: "127.27.40.189" port: 42015 } } peers { permanent_uuid: "d14549caa7c742e6afd21878ebe4aa90" member_type: VOTER last_known_addr { host: "127.27.40.188" port: 34371 } } }
I20260809 03:37:03.299016 28947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12df6e9e434c49a49f0e961705387c25 [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:03.298007 28986 mvcc.cc:205] Tried to move back new op lower bound from 7316466168975405056 to 7316466168775159808. Current Snapshot: MvccSnapshot[applied={T|T < 7316466168811638784}]
W20260809 03:37:03.318797 28792 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:03.317122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54118 (local address 127.27.40.189:42015)
0809 03:37:03.317166 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.317168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.318123 (+ 955us) server_negotiation.cc:407] Connection header received
0809 03:37:03.318223 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.318224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.318235 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.318264 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.318736 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54118: BlockingRecv error: recv got EOF from 127.0.0.1:54118 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 03:37:03.319722 28675 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:03.317498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56974 (local address 127.27.40.190:44923)
0809 03:37:03.317585 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.317587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.319176 (+ 1589us) server_negotiation.cc:407] Connection header received
0809 03:37:03.319267 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.319268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.319279 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.319306 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.319663 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56974: BlockingRecv error: recv got EOF from 127.0.0.1:56974 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 03:37:03.320668 28903 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:03.317865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51810 (local address 127.27.40.188:34371)
0809 03:37:03.317903 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.317905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.320031 (+ 2126us) server_negotiation.cc:407] Connection header received
0809 03:37:03.320202 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.320203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.320213 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.320243 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.320605 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51810: BlockingRecv error: recv got EOF from 127.0.0.1:51810 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 03:37:03.323235 28792 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:03.322400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54132 (local address 127.27.40.189:42015)
0809 03:37:03.322438 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.322440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.322714 (+ 274us) server_negotiation.cc:407] Connection header received
0809 03:37:03.322822 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.322823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.322841 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.322872 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.323191 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54132: BlockingRecv error: recv got EOF from 127.0.0.1:54132 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 03:37:03.326213 28792 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:03.325442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54140 (local address 127.27.40.189:42015)
0809 03:37:03.325473 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.325475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.325742 (+ 267us) server_negotiation.cc:407] Connection header received
0809 03:37:03.325837 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.325837 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.325847 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.325875 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.326175 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54140: BlockingRecv error: recv got EOF from 127.0.0.1:54140 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 03:37:03.334234 28792 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:03.329419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54142 (local address 127.27.40.189:42015)
0809 03:37:03.329473 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.329475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.333728 (+ 4253us) server_negotiation.cc:407] Connection header received
0809 03:37:03.333827 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.333828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.333838 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.333869 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.334189 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54142: BlockingRecv error: recv got EOF from 127.0.0.1:54142 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 03:37:03.340258 28792 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:03.339425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54150 (local address 127.27.40.189:42015)
0809 03:37:03.339463 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.339465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.339741 (+ 276us) server_negotiation.cc:407] Connection header received
0809 03:37:03.339840 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.339841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.339852 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.339882 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.340213 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54150: BlockingRecv error: recv got EOF from 127.0.0.1:54150 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 03:37:03.350338 28792 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:03.349459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54156 (local address 127.27.40.189:42015)
0809 03:37:03.349496 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.349498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.349800 (+ 302us) server_negotiation.cc:407] Connection header received
0809 03:37:03.349901 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.349902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.349913 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.349942 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.350277 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54156: BlockingRecv error: recv got EOF from 127.0.0.1:54156 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 03:37:03.368570 28792 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:03.367685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54172 (local address 127.27.40.189:42015)
0809 03:37:03.367729 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.367731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.368006 (+ 275us) server_negotiation.cc:407] Connection header received
0809 03:37:03.368109 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.368110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.368121 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.368151 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.368505 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54172: BlockingRecv error: recv got EOF from 127.0.0.1:54172 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 03:37:03.402994 28792 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:03.402185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54182 (local address 127.27.40.189:42015)
0809 03:37:03.402239 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.402241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.402384 (+ 143us) server_negotiation.cc:407] Connection header received
0809 03:37:03.402463 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.402464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.402476 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.402565 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.402935 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54182: BlockingRecv error: recv got EOF from 127.0.0.1:54182 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260809 03:37:03.468941 28792 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:03.468199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54192 (local address 127.27.40.189:42015)
0809 03:37:03.468260 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.468262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.468432 (+ 170us) server_negotiation.cc:407] Connection header received
0809 03:37:03.468466 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.468467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.468479 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.468538 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.468878 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54192: BlockingRecv error: recv got EOF from 127.0.0.1:54192 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 03:37:03.598847 28792 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:03.598153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54208 (local address 127.27.40.189:42015)
0809 03:37:03.598231 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.598233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.598321 (+ 88us) server_negotiation.cc:407] Connection header received
0809 03:37:03.598372 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.598373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.598384 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.598450 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.598771 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54208: BlockingRecv error: recv got EOF from 127.0.0.1:54208 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 03:37:03.856822 28792 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:03.856153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54210 (local address 127.27.40.189:42015)
0809 03:37:03.856239 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:03.856241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:03.856252 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:37:03.856287 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:03.856288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:03.856300 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:03.856355 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:03.856741 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54210: BlockingRecv error: recv got EOF from 127.0.0.1:54210 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:37:04.232103 28943 catalog_manager.cc:1259] Loaded cluster ID: 2c104ce2fa3f4064bb7c039b66663324
I20260809 03:37:04.232189 28943 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90: loading cluster ID for follower catalog manager: success
I20260809 03:37:04.232981 28943 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90: acquiring CA information for follower catalog manager: success
I20260809 03:37:04.233376 28943 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d14549caa7c742e6afd21878ebe4aa90: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:37:04.293823 28984 catalog_manager.cc:1259] Loaded cluster ID: 2c104ce2fa3f4064bb7c039b66663324
I20260809 03:37:04.293905 28984 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee: loading cluster ID for follower catalog manager: success
I20260809 03:37:04.294770 28984 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee: acquiring CA information for follower catalog manager: success
I20260809 03:37:04.295106 28984 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 29fc40e23f0549d69e9dde39c553b8ee: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:37:04.372941 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:04.370029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54222 (local address 127.27.40.189:42015)
0809 03:37:04.370163 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:04.370165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:04.370283 (+ 118us) server_negotiation.cc:407] Connection header received
0809 03:37:04.370366 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:04.370366 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:04.370452 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:04.370545 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:04.370720 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:04.371236 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:04.372011 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:04.372243 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:04.372333 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:04.372518 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:04.372648 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:04.372745 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:04.372814 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 03:37:04.372837 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260809 03:37:04.376979 29104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:04.374735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51820 (local address 127.27.40.188:34371)
0809 03:37:04.374854 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:04.374857 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:04.374865 (+ 8us) server_negotiation.cc:407] Connection header received
0809 03:37:04.374912 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:04.374913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:04.374949 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:04.375020 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:04.375112 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:04.375605 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:04.376286 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:04.376476 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:04.376540 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:04.376661 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:04.376755 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:04.376806 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:04.376842 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 03:37:04.376860 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":73,"threads_started":1}
I20260809 03:37:04.381265 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:04.378681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54230 (local address 127.27.40.189:42015)
0809 03:37:04.378732 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:04.378734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:04.378925 (+ 191us) server_negotiation.cc:407] Connection header received
0809 03:37:04.378996 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:04.378997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:04.379008 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:04.379079 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:04.379199 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:04.379713 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:04.380561 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:04.380770 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:04.380844 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:04.381000 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:04.381038 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:04.381089 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:04.381152 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 03:37:04.381172 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 03:37:04.381690 29104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:04.378692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51834 (local address 127.27.40.188:34371)
0809 03:37:04.378739 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:04.378741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:04.378954 (+ 213us) server_negotiation.cc:407] Connection header received
0809 03:37:04.379035 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:04.379036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:04.379045 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:04.379096 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:04.379270 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:04.379725 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:04.380888 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:04.381159 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:04.381199 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:04.381387 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:04.381416 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:04.381449 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:04.381587 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 03:37:04.381610 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 03:37:04.382596 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:04.378736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56990 (local address 127.27.40.190:44923)
0809 03:37:04.378857 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:04.378860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:04.379014 (+ 154us) server_negotiation.cc:407] Connection header received
0809 03:37:04.379061 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:04.379062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:04.379185 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:04.379269 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:04.380006 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:04.380798 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:04.381838 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:04.382138 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:04.382189 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:04.382370 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:04.382447 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:04.382492 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:04.382522 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 03:37:04.382540 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
I20260809 03:37:04.384068 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 28581
I20260809 03:37:04.395525 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 28679
I20260809 03:37:04.411144 27810 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcMSfqS/build/release/bin/kudu with pid 28796
2026-08-09T03:37:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3061 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6266 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (284722 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST